ruby on rails - How long should a Sidekiq job be? -
in sidekiq wiki stated:
- make jobs small , simple
i simple, idempotent , transactional, small? maybe required memory , computing time measure? sidekiq jobs take between 10sec , 30min.
i think 10sec okay, long running task of 30min? loading data of type database memory, run lengthy computations on them , write results. 3 things in 1 worker job.
is fine? or should instead invoke worker job, multiple worker jobs run small computations? problem is, these small computations may need complex hash tables computations , suggested not persist in redis, small simple values.
that depends on how want/have invoke job , whether acceptable takes long.
if run job in shorter intervals takes finish long.
splitting multiple workers here if improve total run time (e.g. if of can run @ same time)
so rule of thumb always: long suits your needs ok.
however:
- on long jobs should consider job might fail mid-execution whatever reason (server crashes etc).
- can still continue lengthy job there or rolled properly?
- also: happens if data changed while executing job?
Comments
Post a Comment