Issue #5564 has been updated by Jon Leighton.


I have just tried applying r33328 to the ruby_1_9_3 branch, and after doing so I still experience this problem. However, I may have done it incorrectly (I don't really know what I am doing).

Regarding the work-around, it is problematic because we don't know until runtime what conversions we want to perform (so just doing "Encoding::Converter.new('ISO-8859-1', 'UTF-8')" won't help if we need to convert Shift_JIS to UTF-8 at runtime, for example). Is there a way around that problem?

Thank you.
----------------------------------------
Backport #5564: 1.9.3 regression with encoding conversion
http://redmine.ruby-lang.org/issues/5564

Author: Jon Leighton
Status: Assigned
Priority: Normal
Assignee: Yuki Sonoda
Category: 
Target version: 


Hello,

We are experiencing a bug in the Rails test suite against ruby 1.9.3.

The bug looks like this:

  1) Error:
test_default_external_works(TestERBTemplate):
ActionView::Template::Error: code converter not found (ISO-8859-1 to UTF-8)
    /home/turnip/Code/rails/actionpack/lib/action_view/template/handlers/erb.rb:83:in `encode!'
    /home/turnip/Code/rails/actionpack/lib/action_view/template/handlers/erb.rb:83:in `call'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:252:in `compile'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:189:in `compile!'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:142:in `block in render'
    /home/turnip/Code/rails/activesupport/lib/active_support/notifications.rb:55:in `instrument'
    /home/turnip/Code/rails/actionpack/lib/action_view/template.rb:141:in `render'
    test/template/template_test.rb:53:in `render'
    test/template/template_test.rb:137:in `block in test_default_external_works'
    test/template/template_test.rb:176:in `with_external_encoding'
    test/template/template_test.rb:135:in `test_default_external_works'
    /home/turnip/.rvm/gems/ruby-1.9.3-p0/gems/mocha-0.10.0/lib/mocha/integration/mini_test/version_230_to_251.rb:28:in `run'

A simplified view of what that code is doing is the following:

 s = "hello \u{fc}mlat"
 s.force_encoding "ISO-8859-1"
 s.encode! 'UTF-8'

However, I have been unable to create any simplified test case for this bug.

What I have been able to find out is the following:

1. Via git-bisect, I tracked down the changed which introduced the bug (it was not present in 1.9.3-preview1). That change was https://github.com/ruby/ruby/commit/05c2cc90895750255f64254f2491ff0ad77cc13a (r32791)
2. If I do some sort of conversion at *load time* in the file action_view/template/handlers/erb.rb, the bug does not occur. For example, if I insert "Encoding::Converter.new('ISO-8859-1', 'UTF-8')" (or the 3 lines above) at the top of the file, I do not experience the bug.

Let me know if I can provide any more information. Also any information about what exactly is triggering this would be very helpful so that we can put a workaround in Rails (if possible).

Thanks


-- 
http://redmine.ruby-lang.org