On Thu, 23 Sep 2004, Eric Hodel wrote:

> DRb::NamedIdConv may be of use also.  It allows clients to die, then come
> back and pick their reference back up.  Also look at TupleSpace, its a good
> place to store things like distributed refcounts since it takes care of
> atomic updates.  (How best to do this is not obvious, and a great book on
> the subject is no longer in print.  Give a holler if you decide to use it
> and need clues.)

eric-

here's what i ended up with.  this is an excerpt from a much larger peice of
code, but it runs standalone and i think it's pretty clear what's happening
but here's a little explanation anyways:

the JobRunnerDaemon exists to that my process can fork without forking - it
forks in another process on my behalf.  other than handling the reaping of the
forked children this is all it's really for.  basically i just need a handle
to track the forked children and a way to reap them in the normal
blocking/non-blocking (WNOHANG, etc) way.

JobRunnerDaemon#gen_runner is the method of interest.  it returns a new
JobRunner but maintains a reference by loading it into a hash (@runners).  all the
various wait methods delete the runner from the hash.  this daemon is used by
one, and only one, process at a time in a single threaded fashoin so i think
this approach is safe:

   - handle on returned DRbUndumped objects is maintained on both client and
     server so nothing should evaporate on me

   - due to the use case (wait) there is a point in the code when i know the
     object can be recycled (reference lost) and this is leveraged by
     automatically disgarding the reference at that point

the code:

   require 'drb'
   require 'detach'
   class JobRunner
   #{{{
     include DRbUndumped
     attr :job
     attr :jid
     attr :cid
     alias pid cid
     attr :shell
     attr :command
     attr :status
     def initialize job
   #{{{
       @status = nil
       @job = job
       @jid = job['jid']
       @command = job['command']
       @shell = job['shell'] || 'bash'
       @r,@w = IO.pipe
       @cid =
         #Util::fork do
         fork do
           @w.close
           STDIN.reopen @r
           if File::basename(@shell) == 'bash' || File::basename(@shell) == 'sh'
             exec [@shell, "__rq_job__#{ @jid }__#{ File.basename(@shell) }__"], '--login'
           else
             exec [@shell, "__rq_job__#{ @jid }__#{ File.basename(@shell) }__"], '-l'
           end
         end
       @r.close
   #}}}
     end
     def run
   #{{{
       @w.puts @command
       @w.close
       self
   #}}}
     end
   #}}}
   end
   class JobRunnerDaemon
   #{{{
     class << self
   #{{{
       def new(*a,&b)
   #{{{
         super(*a,&b).detach(:background=>false)
   #}}}
       end
   #}}}
     end
     attr :runners
     def initialize
   #{{{
       @runners = {}
   #}}}
     end
     def gen_runner(*a,&b)
   #{{{
       r = JobRunner::new(*a,&b)
       @runners[r.pid] = r
       r
   #}}}
     end
     def wait
   #{{{
       pid = Process::wait
       @runners.delete pid
       pid
   #}}}
     end
     def wait2
   #{{{
       pid, status = Process::wait2
       @runners.delete pid
       [pid, status]
   #}}}
     end
     def waitpid pid = -1, flags = 0
   #{{{
       pid = Process::waitpid pid, flags
       @runners.delete pid if pid
       pid
   #}}}
     end
     def waitpid2 pid = -1, flags = 0
   #{{{
       pid, status = Process::waitpid2 pid, flags
       @runners.delete pid if pid
       [pid, status]
   #}}}
     end
   #}}}
   end
   JobRunD = JobRunnerDaemon
   #
   # baby test - watch in top for memory leaks
   #
   if $0 == __FILE__
   #{{{
     STDOUT.sync = true
     d=JobRunnerDaemon::new
     loop do
     #
     # spawn a bunch of jobs on the server
     #
       rand(42).times do
         r=d.gen_runner 'jid'=>42,'command'=>'echo $$'
         r.run
       end
     #
     # reap them
     #
       loop do
         begin
           pid, status = d.waitpid2
           if pid
             p [pid, status]
           else
             break
           end
         rescue Errno::ECHILD
           break
         end
       end
     end
   #}}}
   end


any comments welcome.

kind regards.

-a
--
===============================================================================
| EMAIL   :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE   :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it. 
|   --Dogen
===============================================================================