On Wed, 17 Sep 2008 12:51:14 +1000, James Gray <james / grayproductions.net>  
wrote:

> Can we do:
>
>    ['"', val, '"'].map { |s| s.encode("UTF-16BE") }.join
>
> ?  Yeah, that seems to work.  It sucks, but it works.

Yep, it sure sucks!

I have been doing some more thinking about these ongoing issues....

<soapbox>

Using Ruby SHOULD be making our lives easier, not harder. Other languages  
like Python have taken an easier route to m17n - represent all strings  
internally as unicode codepoints. Then there should never be a need to  
check encoding compatibility, right? I am not saying that this is a  
perfect solution either, by the way. But having to work around this  
"Encoding Compatibility Error" all the time is just a pain for apps which  
need to work in different countries with different locales. Unfortunately  
it is leading me towards the path of having to transcode everything to  
UTF-8, even though in 99% of cases all the data IS going to be compatible  
and be in the user's locale. I don't want so much of my time taken up, and  
be forced to write ugly code to take care of the remaining 1%. Maybe the  
problem is that Ruby is being too generous supporting all these different  
encodings internally! That was one reason why I raised the idea of  
removing UTF-16 & 32 support - at least that way I know that the ASCII  
strings from my program can work with any user data. But then the further  
problem: What if you need to work with (or at least take into account the  
possibility of) 2 or more non-ascii (but ascii compatible) encodings (eg:  
the user's locale & UTF-8)?

What may solve this issue is if Ruby itself would automatically encode  
incompatible strings in a compatible encoding (UTF-8 I guess). The only  
time you should then get "Encoding Compatibility Errors" is when writing  
data to a file or network stream in a certain encoding and a character  
cannot be represented. That's it.

Just a thought...

</soapbox>

Mike