mysql - What is the best data structure for "saved search" feature with daily email notifications? -


the feature works following way:

  • website has users , users can have number of searches saved (e.g. jobs in ny, php jobs, etc). there lot of parameters involved virtually impossible index (i using mysql).
  • every day number of new jobs posted website
  • every 24 hours take jobs posted within last 24 hours , match them against existing job searches , email users matching jobs.

the problem here is high-traffic website , optimistic case (few new jobs posted), takes 10 minutes run search query. there classical solutions problem? we've been using sphinx search-intensive places can't apply here because sphinx won't return results, cut them off eventually. best thing come have search.matched_job_ids column , whenever job posted, match against existing searches , record job id in matched_job_ids column of matched searches. @ end of day email users , truncate column. technically doesn't offer performance improvement spreads load on time executing many smaller search queries rather 1 big query. there better approaches?

each job can described number of parameters - job sphere, job name, salary , on. each parameter has set of predefined values -

  1. job sphere - it,medicine,industry...
  2. job name - programmer, tester, driver...
  3. 10-50 thousands per month, 50-100...
  4. flexy time, full time, freelance...

let's encode saved search. maximal number of values among parameters (i believe job name) base of numeral system. number of parameters - number of digits.

bigint = 2^64-1 = 18 446 744 073 709 551 616 = 20 digits. in normal 10-base system can describe 20-1 (first digit fixed) = 19 parameters each having 10 values. 10 values not enough describing such parameter job name should use 30-60-base system. of course, leads decreasing total number of parameters, thing it's possible describe job 12-15 parameters.

create table savedsearches(code,mail) indexed on (code,mail). index type - primary key.

new job posted:

1) encode programatically.
2) select mail savedsearhes code=calculatedcode. mail in covered index - select sholuld fast enough.
3) send new job selected mails.

important note - 1 parameter - host company of posted job can have values. think should store separately, not in savedsearhes table user don't care company - cares salary, skills e.t.c.

if user wants search not fixed parameter, instance not programmer position tester, team leader have search not single encoded number interval.

my idea assumption, base further investigations))


Comments

Popular posts from this blog

php - Calling a template part from a post -

Firefox SVG shape not printing when it has stroke -

How to mention the localhost in android -