On Jan 2, 2005, at 10:31 AM, Shashank Date wrote:

> I deal with such data almost on a routine basis (although not using 
> MySQL), and use standard "scrubbing techniques":
>
> 1. Try to limit the data in the SELECT, like borrowing from your 
> example:
>
> SELECT * FROM TEST WHERE DT > '1997-01-01';
>
> or
>
>
> SELECT DISTINCT DT FROM TEST;
>
> to see what values show up
>
> 2. Use the datetime / string conversion functions to try to identify 
> the   bad data.
>
> 3. If the data set is small (< 100,000 rows) use CURSOR logic (usually 
> frowned upon) to identify the bad rows
>
> 4. Export the table to a CSV and then use the text filtering tools and 
> then Import it back
>
> Again, not knowing exactly what your situation is, I do not know if 
> all these are even relevant. So I guess, what I am trying to say is, 
> as far as possible try to handle it at the backend BEFORE it reaches 
> DBI.

These scrubbing techniques do help, thanks.

Francis Hwang
http://fhwang.net/