URIS

   http::/codeforpeople.com/lib/ruby/nbfifo

NAME

   nbfifo.rb

SYSNOPSIS

   thread and process safe non-blocking fifos for ruby.

   when using fifos from ruby threads it is extremely easy to block the entire
   process via some operation done on a fifo such as opening it or
   reading/writing to/from it.  rbfifo.rb is a proof-of-concept class showing
   how fifos can be used from a ruby multi-threaded process in a way that
   ensures no threaded operation will block the entire process.  as an added
   benefit the nbfifo class sets up fifos in such a way that they are durable
   across process invocations so that a server may leave a connection open
   while clients come and go and clients do not even block waiting for a server
   to be on the other end of a fifo opened for writing.  another feature is
   that an eof signal can be sent down the fifo to signal end-of-file to
   clients, who will have EOFError raised in there code, however they can
   catch this error and simply reset the fifo to begin reading again.  the
   relationship of readers and writers is such that they may even exist in
   the same process without deadlocking.

CONCEPT

   opening a fifo in 'r+' mode do not block as opening them with only 'r' or
   'w' does.  nbfifo opens all fifos in this way to avoid blocking a process on
   an open.  the reading and writing protocol 'blocks' in the sense that the
   application does not need to handle EAGAIN or some such but not in a way
   that blocks an entire process - the blocking is thread local.  therefore all
   one need do to accomplish non-blocking io is operate in ruby threads (which
   are normally quite prone to a process-blocking operation killing all
   threads).

   the writing is done as follows:

     - never write more than PIPE_MAX bytes at a time to ensure atomicity.  you
       don't worry about this - data is chunked by the api for you.

     - each msg is preceded by a byte indicating it's length.  the length plus
       the msg is less that PIPE_MAX when combined.

     - select is called before writing

     - when select returns the msg is sent

   the reading is done as follows:

     - select is called to avoid reading 'no-data'.  when pipes are opened in
       'r+' mode a blind read with no data waiting will return the emtpy string
       ("") and the application would end up busy-looping on this value unless
       select is used.

     - when select returns the number of bytes is read.  then the message is
       read.

EXAMPLES

   ------------------------------------------
   here's the producer process:
   ------------------------------------------

     jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 > cat producer.rb
     require 'nbfifo'
     STDOUT.sync = true
     $VERBOSE = nil

     #
     # die nicely on getchar
     #
       trap('INT'){ exit }
       Thread::new(){ Thread::main.exit if select [STDIN]}

     #
     # this guy will stay running
     #
       Thread::new{ loop{ puts "random producer thread stays running..."; sleep 1 }}

     #
     # loop doing an open, which does not block, and the sending two virtual
     # doccuments down the pipe but following the first with eof
     #
       loop do
         Thread::new do
           nbfifo = NBFifo::new 'fifo'
           2.times{ nbfifo.send Time::now; sleep 1 }
           nbfifo.send_eof
           2.times{ nbfifo.send Time::now; sleep 1 }
         end.join
       end

   ------------------------------------------
   here's the concumer process
   ------------------------------------------

     jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 > cat consumer.rb
     require 'nbfifo'
     STDOUT.sync = true
     $VERBOSE = nil

     #
     # die nicely on getchar
     #
       trap('INT'){ exit }
       Thread::new(){ Thread::main.exit if select [STDIN]}

     #
     # this guy will stay running
     #
       Thread::new{ loop{ puts "random consumer thread stays running..."; sleep 1 }}

     #
     # just loop opening fifo, which does not block, and reading whatever data
     # can be read until end-of-file is hit.  when it is reset and go again
     #
       loop do
         Thread::new do
           nbfifo = NBFifo::new 'fifo'
           begin
             2.times{ puts nbfifo.recv }
           rescue EOFError
             nbfifo.reset
             retry
           end
         end.join
       end

   ------------------------------------------
   running the consumer
   ------------------------------------------

     jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 > ruby consumer.rb
     random consumer thread stays running...
     Fri Sep 23 15:35:31 MDT 2005
     Fri Sep 23 15:35:32 MDT 2005
     random consumer thread stays running...
     Fri Sep 23 15:35:35 MDT 2005

   ------------------------------------------
   running the producer
   ------------------------------------------

     jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 > sleep 3; ruby producer.rb
     random producer thread stays running...
     random producer thread stays running...
     random producer thread stays running...
     random producer thread stays running...

   ------------------------------------------
   running the consumer after killing it
   ------------------------------------------

     jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 > ruby consumer.rb
     random consumer thread stays running...
     Fri Sep 23 15:43:15 MDT 2005
     Fri Sep 23 15:43:17 MDT 2005
     random consumer thread stays running...
     Fri Sep 23 15:43:26 MDT 2005


   note that killing the consumer and restarting did not affect the producer at
   all.  this is quite different than the way normal fifo operation is - in that
   case the reader must always open the fifo after a writer is on the other end
   and a writer - even if opening using O_NONBLOCK will explode if no reader is
   on the other end.  using this method neither open (on either end) will block
   but subsequent reads/writes will, but in a way that blocks only thread and not
   the entire process.  in fact, you can easily open up a fifo for reading and
   writing in the same process, regardless of the order they are opened, and
   easily write well behaved threads that use this channel to communicate:


   ------------------------------------------
   running a producer and consumer at once
   ------------------------------------------

     jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 > cat producer_and_consumer.rb
     require 'nbfifo'
     STDOUT.sync = true
     $VERBOSE = nil

     trap('INT'){ exit }
     Thread::new(){ Thread::main.exit if select [STDIN]}

     producer =
       Thread::new do
         nbfifo = NBFifo::new 'fifo'
         loop{ nbfifo.send Time::now; sleep 1 }
       end

     consumer =
       Thread::new do
         nbfifo = NBFifo::new 'fifo'
         loop{ puts nbfifo.recv }
       end

     sleep


     jib:~/eg/ruby/nbfifo/nbfifo-0.0.0 > ruby producer_and_consumer.rb
     Fri Sep 23 15:47:13 MDT 2005
     Fri Sep 23 15:47:14 MDT 2005
     Fri Sep 23 15:47:15 MDT 2005
     Fri Sep 23 15:47:16 MDT 2005
     Fri Sep 23 15:47:17 MDT 2005


   note that this program would deadlock instantly using fifos in the normal way
   ('r', or 'w') and would blow up with Errno::ENXIO if File::NONBLOCK were
   specified to the open and the writer thread attempted the open before the
   reader.  the other issue with using File::NONBLOCK for fifos is that reads
   give the empty string ("") whether we're at the end-of-file or if there is
   simply no data and we're forced into a busy loop.


CAVEATS

   like all my code these days - this is experimental.  you io gurus out there
   will no doubt have comments and, for you, this process may seem easy (that's
   you tanaka) but it was giving me fits so i decided to wrap it up.

   i've run the code on linux only.  i'd be grateful for some bsd and solaris
   testers.  also, i played around with gunwin32 utils (for mkfifo) on windows
   and couldn't get it worked - but it was close.  please contact me off/online
   if you have helpful info in this area.


WHY?

   say you were writing a fastcgi replacement and wanted a multi-threaded pure
   ruby server which could handled multiple incming requests without blocking
   the entire process opening/reading/writing to/from fifos.  then you'll
   really need something like this.

AUTHOR

   ara.t.howard / noaa.gov

-a
-- 
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze.  --Nagarjuna
===============================================================================