On Sep 17, 2008, at 9:32 PM, Michael Selig wrote:

> I was only suggesting conversion to Unicode as a way of preventing  
> an error being raised.

Hey, I know character encodings are hard.  I'm still trying to get CSV  
completely converted.  I'm getting closer all the time, but it's been  
tricky for sure.

I'm sure it's a bit of Ruby's fault.  The m17n code is still a little  
raw and I ran into several issues just exploring it.  All of our  
efforts here are making things better though.  Look at how many bugs  
were fixed in the last week just do to emails from you and me.

The other thing that's very important to remember is that character  
encodings are just plain hard to get right.  I think it's a pretty big  
testament that Ruby makes it possible for us to support all these  
encodings now.

I'm definitely in the self-centered-universe camp that thought Unicode  
was best for most things.  I know I would still recommend it in many  
cases, because it's pretty easy to implement and it does work in many  
cases.

However, our Japanese friends are trying to tell us it's not a  
universal solution.  It doesn't always work well for them in  
particular, so they would prefer we make something better.  I for one  
am grateful for them teaching me this new lesson, hard or not.

And if you prefer to do the UTF-8 everywhere strategy, you can,  
right?  Transcode everything to UTF-8 when it comes in and then you  
can pretend it's all UTF-8 (because it is!), right?  Don't we have the  
best of both worlds now?

James Edward Gray II