On Sun, May 5, 2013 at 12:56 AM, Tanaka Akira <akr / fsij.org> wrote: > 2013/5/5 Nikolai Weibull <now / disu.se>: >> Why isn”Ēt setlocale called with LC_ALL instead of LC_CTYPE in main.c? > Changing setlocale(LC_CTYPE, "") to setlocale(LC_ALL, "") still cause problems. It causes problems that are based precisely on the fact that they”Ēre locale sensitive. > make check with changed Ruby on GNU/Linux fails as follows with > tr_TR.UTF-8 locale. > (tr_TR locale changes case related bahavior. [ruby-core:14662]) > % LANG=tr_TR.UTF-8 make check > [ 247/13312] DL::TestDL#test_call_double = 0.09 s > 1) Failure: > DL::TestDL#test_call_double [/home/akr/ruby/tst1/ruby/test/dl/test_dl2.rb:90]: > Expected |0.1 - 0.0| (0.1) to be <= 0.001. Bugs because atof is called, which is locale sensitive. > [ 279/13312] DL::TestFunc#test_strtod = 0.09 s > 2) Failure: > DL::TestFunc#test_strtod [/home/akr/ruby/tst1/ruby/test/dl/test_func.rb:152]: > Expected |12.34 - 12.0| (0.33999999999999986) to be <= 0.001. Same here, but with locale-sensitive function strtod. > [ 439/13312] Fiddle::TestFunc#test_strtod = 0.09 s > 3) Failure: > Fiddle::TestFunc#test_strtod > [/home/akr/ruby/tst1/ruby/test/fiddle/test_func.rb:56]: > Expected |12.34 - 12.0| (0.33999999999999986) to be <= 0.001. And again. > [ 2133/13312] TestArgf#test_unreadable = 0.00 s > 4) Error: > TestArgf#test_unreadable: > ArgumentError: invalid byte sequence in US-ASCII > /home/akr/ruby/tst1/ruby/test/ruby/test_argf.rb:803:in `block in > test_unreadable' This seems to be a problem with paths being locale sensitive and therefore encoding sensitive, but not created with the right encoding. This is a more complicated problem than the previous ones, but it surely is a bug in Ruby, not a reason not to enable locale support in Ruby. > [ 2379/13312] TestBigDecimal#test_BigMath_exp = 0.00 s > 5) Failure: > TestBigDecimal#test_BigMath_exp > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1280]: > Expected |485165195.4097903 - 0.48516519540979027797E9| > (485165195.4097903) to be <= 0.48516519540979027797E6. > > [ 2383/13312] TestBigDecimal#test_BigMath_log_with_2 = 0.00 s > 6) Failure: > TestBigDecimal#test_BigMath_log_with_2 > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1365]: > Expected |0.6931471805599453 - > 0.6931471805599453094172321214581765674099931010273832E0| > (0.6931471805599453) to be <= 0.001. > > [ 2384/13312] TestBigDecimal#test_BigMath_log_with_42 = 0.01 s > 7) Failure: > TestBigDecimal#test_BigMath_log_with_42 > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1375]: > Expected |3.7376696182833684 - > 0.3737669618283368305917830101823881909945797027394132E1| > (3.7376696182833684) to be <= 0.001. > > [ 2386/13312] TestBigDecimal#test_BigMath_log_with_exp_1 = 0.00 s > 8) Failure: > TestBigDecimal#test_BigMath_log_with_exp_1 > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1361]: > Expected |1.0 - > 0.10000000000000000000146094228462874163031819506060204E1| (1.0) to be > <= 0.001. > > [ 2393/13312] TestBigDecimal#test_BigMath_log_with_reciprocal_of_42 = > 0.00 s > 9) Failure: > TestBigDecimal#test_BigMath_log_with_reciprocal_of_42 > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1381]: > Expected |-96.70857390574992 - > -0.96708573905749918728755641096743294420914068228418976E2| > (96.70857390574992) to be <= 0.001. > > [ 2425/13312] TestBigDecimal#test_exp_with_1 = 0.00 s > 10) Failure: > TestBigDecimal#test_exp_with_1 > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1275]: > Expected |2.718281828459045 - 0.27182818284590452354E1| > (2.718281828459045) to be <= 0.002718281828459045. > > [ 2469/13312] TestBigDecimal#test_power_of_three = 0.00 s > 11) Failure: > TestBigDecimal#test_power_of_three > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:1007]: > Expected |0.012345679012345678 - > 0.12345679012345679012345679012345679012345679012345679012345679E-1| > (0.012345679012345678) to be <= 0.001. > > [ 2488/13312] TestBigDecimal#test_sqrt_bigdecimal = 0.00 s > 12) Failure: > TestBigDecimal#test_sqrt_bigdecimal > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:786]: > Expected |0.3 - 0.3E0| (0.3) to be <= 0.001. > > [ 2495/13312] TestBigDecimal#test_to_f = 0.00 s > 13) Failure: > TestBigDecimal#test_to_f > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigdecimal.rb:557]: > <1.0> expected but was > <0.0>. > > [ 2510/13312] TestBigMath#test_atan = 0.00 s > 14) Failure: > TestBigMath#test_atan > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:57]: > Expected |0.7853981633974483 - > 0.78539816339744830961566084581987572097996482338035675E0| > (0.7853981633974483) to be <= 0.001. > > [ 2511/13312] TestBigMath#test_const = 0.00 s > 15) Failure: > TestBigMath#test_const > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:13]: > Expected |3.141592653589793 - > 0.3141592653589793238462643383279502883919859293521427E1| > (3.141592653589793) to be <= 0.001. > > [ 2512/13312] TestBigMath#test_cos = 0.00 s > 16) Failure: > TestBigMath#test_cos > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:42]: > Expected |1.0 - 0.1E1| (1.0) to be <= 0.001. > > [ 2513/13312] TestBigMath#test_sin = 0.00 s > 17) Failure: > TestBigMath#test_sin > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:29]: > Expected |0.7071067811865476 - > 0.70710678118654752440084436210484903917782034136995079E0| > (0.7071067811865476) to be <= 0.001. > > [ 2514/13312] TestBigMath#test_sqrt = 0.00 s > 18) Failure: > TestBigMath#test_sqrt > [/home/akr/ruby/tst1/ruby/test/bigdecimal/test_bigmath.rb:18]: > Expected |1.4142135623730951 - > 0.141421356237309504880168872420969807825E1| (1.4142135623730951) to > be <= 0.001. These I can”Ēt understand with only a cursory glance. Do you know why they”Ēre failing? I see that BigDecimal_to_f uses strtod (which is locale dependent and should therefore most likely not be used) and perhaps that”Ēs the culprit? > [ 4903/13312] TestGemExtCmakeBuilder#test_self_build = 0.02 s > 19) Skipped: > TestGemExtCmakeBuilder#test_self_build > [/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_ext_cmake_builder.rb:11]: > cmake not present This fails for me as I don”Ēt have cmake. > [ 4904/13312] TestGemExtCmakeBuilder#test_self_build_fail = 0.02 s > 20) Skipped: > TestGemExtCmakeBuilder#test_self_build_fail > [/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_ext_cmake_builder.rb:11]: > cmake not present > > [ 4905/13312] TestGemExtCmakeBuilder#test_self_build_has_makefile = 0.03 s > 21) Skipped: > TestGemExtCmakeBuilder#test_self_build_has_makefile > [/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_ext_cmake_builder.rb:11]: > cmake not present These two as well. > [ 4994/13312] TestGemInstaller#test_build_extensions_extconf_bad = > 0.10 s > 22) Failure: > TestGemInstaller#test_build_extensions_extconf_bad > [/home/akr/ruby/tst1/ruby/test/rubygems/test_gem_installer.rb:86]: > Expected /\/home\/akr\/ruby\/tst1\/ruby\/ruby: No such file/ to match > "/home/akr/ruby/tst1/ruby/ruby > extconf.rb\n/home/akr/ruby/tst1/ruby/ruby: Byle bir dosya ya da dizin > yok -- extconf.rb (LoadError)\n". OK, this fails because the error message generated by strerror is being localized. If this is really a problem beyond a test that depends on the locale being en_US, then perhaps exclude LC_MESSAGES from setlocale. > [ 7181/13312] TestMiniTestUnitTestCase#test_capture_subprocess_io = > 0.00 s > 23) Skipped: > TestMiniTestUnitTestCase#test_capture_subprocess_io > [/home/akr/ruby/tst1/ruby/test/minitest/test_minitest_unit.rb:1401]: > Dunno why but the parallel run of this fails Hard-coded skip, so this isn”Ēt related to locales. > [ 8173/13312] TestProcess#test_execopts_env = 0.07 s > 24) Error: > TestProcess#test_execopts_env: > ArgumentError: invalid byte sequence in US-ASCII > /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:260:in `sub' > /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:260:in `block > in test_execopts_env' > /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:33:in `block (2 > levels) in with_tmpchdir' > /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:32:in `chdir' > /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:32:in `block in > with_tmpchdir' > /home/akr/ruby/tst1/ruby/lib/tmpdir.rb:88:in `mktmpdir' > /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:30:in `with_tmpchdir' > /home/akr/ruby/tst1/ruby/test/ruby/test_process.rb:251:in > `test_execopts_env' This is also related to strerror being used by Errno error messages. Strerror returns a localized message if LC_MESSAGES is included in setlocale() and in the codeset/encoding of the locale, if so. > [10528/13312] TestRakeFunctional#test_signal_propagation_in_testsTerminated > = 0.08 s > 25) Skipped: > TestRakeFunctional#test_signal_propagation_in_tests > [/home/akr/ruby/tst1/ruby/test/rake/test_rake_functional.rb:436]: > Signal detect seems broken on this system I don”Ēt know why this fails, but perhaps it”Ēs not locale dependent? > It seems four years are not enough to make ruby robust with locales. > I think it is acceptable that make ruby more robust. > (i.e. fixing above failures and errors are ok.) > But changing setlocale invocation to LC_ALL is too dangerous. > The change affects not only ruby itself but also other extension libraries. Precisely my point and precisely why setlocale should be called with LC_ALL. It”Ēs impossible to write extensions tat use locale-dependent functionality unless Ruby actively supports it. Your argument held equally true before setlocale(LC_CTYPE) was added, but still it was added.