On Jul 28, 2006, at 3:33 PM, Ashley Moran wrote:

> I looked at the source of the script today, and I made these changes:
>
> - use FasterCSV instead of CSV

Fair warning, I'm coming into this conversation late and I haven't  
read all that came before this.  However, if you are using FasterCSV...

> Unfortunately I'm still stuck with a nasty "rows.each { |row|  
> row.each { |col|
> col.strip! } }" type section, to fix the poor quality of the data,  
> which
> would take a lot of time going through all the fields to thin out.

FasterCS can convert fields as they are read.  I'm not sure if this  
will be faster, but it may be worth a shot.  See the :converters  
argument to FasterCSV::new:

http://fastercsv.rubyforge.org/

James Edward Gray II