On 01/07/2010 08:01 PM, Phillip Gawlowski wrote:
> On 07.01.2010 19:50, Iki Baz Castillo wrote:
> 
>>> pipe.write unless pipe.full?
>> Unfortunatelly #full? is not a method of File :(
> 
> Well, yes, you'd have to implement the method (or something like it) 
> yourself. ;)
> 
>> Note that I'm using a fifo file (created with "mkfifo file") so it is not
>> "stored" in the filesystem. Instead it's just a communication between two
>> processes at SO level via SO's buffers.
> 
> Yeah, I gathered that from your other posts. The general point, though, 
> still applies: check the pipe's size, and if it grows too large, spin 
> off a new reading thread.

That's something different than you proposed initially, isn't it?  This 
approach (increasing the number of readers if the pipe fills too fast) 
is better because it regulates read performance according to load.

> pipe.write unless pipe.full?
> 
> i.e. check if your pipe hits a set limit on disk, and generate an 
> exception if the pipe_file reaches (or is close to reaching) the limit. 
> You could then buffer the data to be written until an additional (or 
> new) reading thread has started.

IMHO this approach (local buffering if the pipe cannot be written to) is ot really helping because the pipe *is* a buffer already.  In other 
words, the same effect will happen - only later.  The only argument in 
favor of additional buffering I can see is less lock contention: if 
every writer process has multiple threads that want to write to the 
buffer, they could instead write to a Queue internally and a single 
reader could read from that local queue and write to the global queue. 
That would reduce the number of writers that compete for locks on the 
global queue.  Whether that is performant or not would need to be tested.Nevertheless I would start with a simple solution, monitor its 
performance and change the implementation if it does not scale well 
enough.  Often simple solutions work surprisingly well... :-)

Kind regards

	robert

-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/