On Thu, 13 May 2004 01:53:53 +0900, Ara.T.Howard wrote
 
> however, it __would__ be very cool if i could use it: the project is 
> a distributed job manager (grid) that makes use of an on nfs disk 
> priority job queue (pstore or yamlstore and accessed safely by means 
> of my Lockfile class) and n consumer processes competing to run jobs 
> from the queue.  the advantage of using yaml is that standard tools 
> (eg. grep) might be used to check for the status of jobs... also it 
> might make possible the construction of tool sets that utilize each 
> others stdout/in a bit easier, eg.

Ara, I'm working on something similar right now, using a different 
architecture.

I'm using a combination of DRb, Rinda/RingServer, and a TupleSpace to 
receive a queue of jobs that the consumers processes then take off of the 
tuplespace.  I'm planning on coupling the tuplespace to a persistent store 
so that if something happens to the process that holds the tuplespace, the 
queue isn't lost and can be restored when the tuplespace server restarts.

In my tests so far it seems to be a stable and flexible way to queue jobs 
since I can make use of objects in the queue to pass information from where 
the job is generate to the consumer process.  It, however, has the same 
problem that your PStore based queue does in that simple tools like grep can 
not be used to check the status of the queue.  Specialized tools have to be 
written to do that.


Kirk Haines

P.S. Is the weather as cold and wet down in Boulder today as it is up here 
in Wyoming?  Good day to be inside and writing some Ruby code, I think.