Austin Ziegler wrote:
> On Thu, 3 Feb 2005 11:18:18 +0900, jm <jeffm / ghostgun.com> wrote:
> 
>>Processing flow data from a router. The script caches user information
>>from a database then processes a 25-30MB flow file captured from a
>>router using flow-tools. This takes about 5 minutes on a 2.4GHz pentium
>>4 unloaded by any other process running at 98% utilisation
>>continuously. To put this into context each flow file is only 15
>>minutes worth of data and the current perl version does it in a bit
>>over 2 minutes. This perl version is showing it's lack of design in a
>>variety of ways including the nightmare of trying to add features it
>>was never designed to support. So this was a good opportunity to
>>rewrite it in ruby to make it more maintainable, etc.
>>
>>While 5 minutes in within the time constraint that is on an unloaded
>>machine and the machine it's destined for has other processes sharing
>>the CPU.
> 
> 
> Could your script be doing things that could be improved in performance?

Well, I think Ruby/DBI is not the fastest. It parses (and splits) each 
SQL statement, then joins it back into a string, even if you don't use 
'?' parameter markers. This should be delayed and omitted if no 
parameters were given. Not sure whether this is the reason for the slowness.

Regards,

   Michael