--0016e68debb795cb520475ff466a
Content-Type: text/plain; charset=ISO-8859-1

On Thu, Oct 15, 2009 at 10:51 AM, Chris Wailes <chris.wailes / gmail.com>wrote:

> What I'm creating is a job queue.  A thread from a central thread pool
> grabs a job and then executes it.  In very special circumstances, when a
> job on the queue is waiting for a response from another job on the
> queue, the job needs to pause and be put back onto the queue to wait for
> a value.


(excuse the shameless self-promotion, but...)

You might look at Revactor, or the actor model in general:

http://revactor.org/

Revactor is a Fiber-based implementation of the Actor model, full of
baked-in sync/async I/O.

Here's an example of doing scatter/gather-type processing of a job queue.
In this case, there are multiple workers handling synchronous I/O events,
pulling from a central job queue:

http://github.com/tarcieri/revactor/blob/master/lib/revactor/http_fetcher.rb

What type of jobs are you trying to run?  Are they I/O bound or CPU bound?

Revactor can help you in the I/O bound case but not in the CPU bound case.
However there are many tools available for doing scatter/gather job queues
among a pool of Ruby processes.  That will let you distribute the load of a
job across multiple CPU cores.

-- 
Tony Arcieri
Medioh/Nagravision

--0016e68debb795cb520475ff466a--