On 01/07/2010 08:01 PM, Phillip Gawlowski wrote:
> On 07.01.2010 19:50, I=F1aki Baz Castillo wrote:
>=20
>>> pipe.write unless pipe.full?
>> Unfortunatelly #full? is not a method of File :(
>=20
> Well, yes, you'd have to implement the method (or something like it)=20
> yourself. ;)
>=20
>> Note that I'm using a fifo file (created with "mkfifo file") so it is =
not
>> "stored" in the filesystem. Instead it's just a communication between =
two
>> processes at SO level via SO's buffers.
>=20
> Yeah, I gathered that from your other posts. The general point, though,=
=20
> still applies: check the pipe's size, and if it grows too large, spin=20
> off a new reading thread.

That's something different than you proposed initially, isn't it?  This=20
approach (increasing the number of readers if the pipe fills too fast)=20
is better because it regulates read performance according to load.

> pipe.write unless pipe.full?
>=20
> i.e. check if your pipe hits a set limit on disk, and generate an=20
> exception if the pipe_file reaches (or is close to reaching) the limit.=

>=20
> You could then buffer the data to be written until an additional (or=20
> new) reading thread has started.

IMHO this approach (local buffering if the pipe cannot be written to) is =

not really helping because the pipe *is* a buffer already.  In other=20
words, the same effect will happen - only later.  The only argument in=20
favor of additional buffering I can see is less lock contention: if=20
every writer process has multiple threads that want to write to the=20
buffer, they could instead write to a Queue internally and a single=20
reader could read from that local queue and write to the global queue.=20
That would reduce the number of writers that compete for locks on the=20
global queue.  Whether that is performant or not would need to be tested.=


Nevertheless I would start with a simple solution, monitor its=20
performance and change the implementation if it does not scale well=20
enough.  Often simple solutions work surprisingly well... :-)

Kind regards

	robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/