On 5/27/07, Sy Ys <subsume / gmail.com> wrote:
> I'm making a site which allows subscribers to scrape their unnamed
> social network profile for changes regularly using BackgrounDrb. I can:
>
> A) Create 1 worker which will update all subscriber profiles at once. I
> like this because its simple. I don't like this because it will create a
> server-intensive traffic spike.
>
> B) Instantiate a new worker which will run every 24 hours after a person
> signs up. I like this because it eliminates the spike but still updates
> everyone. But I'm not sure what effect having an worker running for each
> user will have on server memory.
>
> C) Other?

C) Create a pool of n workers which will pull jobs from a queue (maybe
also look at something like Ruby Queue[1]).

[1] http://codeforpeople.com/lib/ruby/rq/rq-3.1.0/