On Thursday, 17 April 2003 at 23:02:08 +0900, Brian Candler wrote:
> On Thu, Apr 17, 2003 at 10:11:55PM +0900, David King Landrith wrote:
> > >Do you think that mmap will get me speeds near cp?
> > 
> I have the source code for FreeBSD's cp on my machine. If the file is under
> 8M it mmap's the source for reading, and calls write() repeatedly on the
> target file until it has been written completely. Otherwise it goes into a
> simple read() / write() loop with a buffer of MAXBSIZE, which is 65536
> 
> There is a comment which says:
> 
>         /*
>          * Mmap and write if less than 8M (the limit is so we don't totally
>          * trash memory on big files.  This is really a minor hack, but it
>          * wins some CPU back.
>          */
> 
> I guess the main difference is that instead of two syscalls for each 64K of
> data, you have one syscall for whatever the maximum size write() will accept
> at once.
> 

Thanks for that info. From the above, I don't think I am going to worry
about it.

> If you're doing things where this is significant, then Ruby is probably the
> wrong language for you - unless you write your entire read-process-write
> loop in C as a single function, in which case using Ruby as a convenient way
> to start it is fine. If it calls 'yield' on each line of the input then that
> is almost certain to dominate...
> 

Our performance is significant mostly from a human perspective and not from
any real-time processing constraints. We now use Perl for this
processing and it is very hard to beat with just read/write.

I have put the read part in RubyC and we have now surpassed Perl.
Plus, we get the added benefit of a common parser library and
much cleaner code.

Beating Perl was the main impetus here. Not driving the time down
to tens of seconds. If that was important, we would have chosen
assembly. :)

It's the old trade-off between performance, maintainability and
development time. With ruby and a C library to speed up the routine,
we have lowered the cost in each category. A big win in my book.

-- 
Jim Freeze
----------
You know you're a little fat if you have stretch marks on your car.
		-- Cyrus, Chicago Reader 1/22/82