Hi,

----- Original Message ----- 
From: "James Gray" <james / grayproductions.net>
To: <ruby-core / ruby-lang.org>
Sent: Thursday, September 18, 2008 1:43 PM

> The other thing that's very important to remember is that character
> encodings are just plain hard to get right.  I think it's a pretty big
> testament that Ruby makes it possible for us to support all these
> encodings now.

> However, our Japanese friends are trying to tell us it's not a
> universal solution.  It doesn't always work well for them in
> particular, so they would prefer we make something better.  I for one
> am grateful for them teaching me this new lesson, hard or not.

I agree with you. And if I am coming across as being a jerk or too dogmatic, 
I don't mean to be!
I was trying to make some constructive suggestions (some may have been 
misguided :-) so that Ruby can meet my needs better, and I think my needs 
may be quite common as software and data sources become more international. 
The intent is to make Ruby's character encoding issues as transparent as 
possible without losing the specific Japanese/Chinese/Welsh(?) requirements 
(now that I understand them a bit better), and of course to provoke further 
discussion about the issue.

I think other people will soon face the sorts of problems you and I have 
been hitting over the past couple of weeks. It has been a very good learning 
experience for me also!

> And if you prefer to do the UTF-8 everywhere strategy, you can,
> right?  Transcode everything to UTF-8 when it comes in and then you
> can pretend it's all UTF-8 (because it is!), right?  Don't we have the
> best of both worlds now?

Nearly! Even if I transcode to UTF-8, I still have to make sure I do it at 
every interface, and that includes to Ruby's standard classes as well - not 
just IO. So I'll still have to check encodings of strings returned from 
network classes, and that's something I don't think I need to do with other 
languages that support Unicode, because there is only one internal string 
representation. Also testing that I have got it all right may be a nightmare 
(not that I am anywhere near that stage yet!). It would be so much nicer to 
have Ruby handle most of this for me.

I am a relatively recent convert to Ruby, mainly from Python. This means I 
am constantly thinking "could I do this easier/better with Python?". And the 
answer to that question for this latest project seems to be leaning towards 
"yes" unfortunately, and I'd like to say a definitive "no", because I like 
so many things about Ruby a great deal!

Cheers
Mike.