Hi,

James Gray wrote:
> On Sep 16, 2008, at 11:21 PM, Michael Selig wrote:
> 
>> Using Ruby SHOULD be making our lives easier, not harder. Other 
>> languages like Python have taken an easier route to m17n - represent 
>> all strings internally as unicode codepoints. Then there should never 
>> be a need to check encoding compatibility, right? I am not saying that 
>> this is a perfect solution either, by the way. But having to work 
>> around this "Encoding Compatibility Error" all the time is just a pain 
>> for apps which need to work in different countries with different 
>> locales. Unfortunately it is leading me towards the path of having to 
>> transcode everything to UTF-8, even though in 99% of cases all the 
>> data IS going to be compatible and be in the user's locale.
> 
> I believe Matz has said in the past that transcoding is what they are 
> trying to avoid in general.  You can loose data that way and thus the 
> core team doesn't favor it.  (I hope I got that right.  It's from 
> memory, so don't blame me for putting words in Matz's mouth.)
> 
> Besides, I'm not sure if it's the characters I have tried or just that 
> Ruby's transcoding still needs work, but I've tried converting some 
> Shift_JIS to UTF-8 that it just couldn't handle.  We would have to have 
> a better conversion rate to support a strategy like this.

We can convert "all Shift_JIS characters" to Unicode now.
But current problem is, there are some mappings Shift_JIS and Unicode conversion.
Once you convert data from Shift_JIS to Unicode, true meaning of some characters
may be lost forever. (e.g. YEN SIGN Problem)

If we develop "a better" conversion, this problem will be more complex.

-- 
NARUSE, Yui  <naruse / airemix.jp>