Bugs item #9300, was opened at 2007-03-15 02:35
You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=1698&aid=9300&group_id=426

Category: Core
Group: 1.8.6
Status: Closed
Resolution: Accepted
Priority: 3
Submitted By: Jeff Xiong (gigix)
Assigned to: Nathaniel Talbott (ntalbott)
Summary: exit called from at_exit blocks does not properly return the exit code

Initial Comment:
If create a sub-process, and run failing tests in it, the sub-process will not exit with error code as expected: the exit code is always 0. 

Ubuntu Linux 7.04
Ruby 1.8.6
Rails 1.2.3

----------------------------------------------------------------------

Comment By: Brian Takita (btakita)
Date: 2007-03-26 12:54

Message:
Nm. I didn't read the other comments.

----------------------------------------------------------------------

Comment By: Brian Takita (btakita)
Date: 2007-03-26 11:27

Message:
Heres a simpler reproduction.

at_exit do
  exit 1
end


----------------------------------------------------------------------

Comment By: Brian Takita (btakita)
Date: 2007-03-26 10:53

Message:
I'm also getting this issue on Ubuntu 6.0.4 and Ruby 1.8.6.

Rails is not loaded.

Here is my reproduction:

require 'test/unit'
at_exit do
  unless $! || Test::Unit.run?
    begin
      result = Test::Unit::AutoRunner.run
      exit result
    rescue => e
      $stderr.puts e.message
      $stderr.puts e.backtrace
      exit 1
    end
  end
end

class Test::Unit::AutoRunner
  def run
    raise "Foobar"
  end
end

We overrode at_exit to properly handle exceptions in Autorunner. Btw, this also has an exit code of 0.

require 'test/unit'
class Test::Unit::AutoRunner
  def run
    return false
  end
end


----------------------------------------------------------------------

Comment By: Nathaniel Talbott (ntalbott)
Date: 2007-03-26 09:17

Message:
Nobu has checked in a fix (and tests) for this now, so I'm closing it. The issue of a maintenance release is still open, and is being discussed on ruby-core.

----------------------------------------------------------------------

Comment By: Nathaniel Talbott (ntalbott)
Date: 2007-03-23 07:32

Message:
Nobu posted a fix in ruby-core:10748, and I posted a failing test case in ruby-core:10760. I will close this bug once Nobu has committed the fix.

----------------------------------------------------------------------

Comment By: Alexey Verkhovsky (alexeyv)
Date: 2007-03-22 21:27

Message:
Finally, the same problem is reproducible on the current 1.8 branch

----------------------------------------------------------------------

Comment By: Alexey Verkhovsky (alexeyv)
Date: 2007-03-22 20:39

Message:
OK, take Test::Unit out of the equation, too. Ruby 1.8.6 ignores the argument to exit(), if inside the at_exit block.

alexeyv@alexeyv-laptop:~/foo$ cat at_exit_false.rb 
puts 'here'

at_exit do
  exit false
end

alexeyv@alexeyv-laptop:~/foo$ ruby at_exit_false.rb 
here
alexeyv@alexeyv-laptop:~/foo$ echo $?
0


----------------------------------------------------------------------

Comment By: Alexey Verkhovsky (alexeyv)
Date: 2007-03-22 20:25

Message:
Hmm, I don't see a way to attach a file to a comment.

Here is the test:

alexeyv@alexeyv-laptop:~/foo$ cat test.rb 
require 'test/unit'

class FooTest < Test::Unit::TestCase
  def test_that_fails
    fail 'oops'
  end
end

Here is the console session that demonstrates the problem:

alexeyv@alexeyv-laptop:~$ cd foo
alexeyv@alexeyv-laptop:~/foo$ which ruby
/usr/bin/ruby
alexeyv@alexeyv-laptop:~/foo$ 'ruby -v
> 
alexeyv@alexeyv-laptop:~/foo$ ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]
alexeyv@alexeyv-laptop:~/foo$ ./run_test_and_print_exit_code.sh 
Loaded suite test
Started
E
Finished in 0.000679 seconds.

  1) Error:
test_that_fails(FooTest):
RuntimeError: oops
    test.rb:5:in `test_that_fails'

1 tests, 0 assertions, 0 failures, 1 errors
1
alexeyv@alexeyv-laptop:~/foo$ switch_ruby
alexeyv@alexeyv-laptop:~/foo$ which ruby
/home/alexeyv/ruby1.8.6/bin/ruby
alexeyv@alexeyv-laptop:~/foo$ ruby -v
ruby 1.8.6 (2007-03-05 patchlevel 0) [i686-linux]
alexeyv@alexeyv-laptop:~/foo$ ./run_test_and_print_exit_code.sh 
Loaded suite test
Started
E
Finished in 0.000614 seconds.

  1) Error:
test_that_fails(FooTest):
RuntimeError: oops
    test.rb:5:in `test_that_fails'

1 tests, 0 assertions, 0 failures, 1 errors
0
^^^
THERE SHOULD BE '1' INSTEAD OF '0' HERE

----------------------------------------------------------------------

Comment By: Alexey Verkhovsky (alexeyv)
Date: 2007-03-22 20:20

Message:
By the way, note that the issue Jeff talks about is a zero exit code returned by a FAILING test. Your printout seems to be checking exit codes of passing tests.

----------------------------------------------------------------------

Comment By: Alexey Verkhovsky (alexeyv)
Date: 2007-03-22 20:17

Message:
Nathaniel, hi,

I'm reproducing the same issue, too. With Ruby 1.8.6 release. 

Attached archive (foo.tgz) contains a very simple failing test, and a script that runs it and prints out the exit code. No Rails, Rake or any other third-party stuff involved. At all.

I'm going to try figure out the reason now.

Alex Verkhovsky

----------------------------------------------------------------------

Comment By: Nathaniel Talbott (ntalbott)
Date: 2007-03-21 19:50

Message:
It's working for me, under 1.8.4, 1.8.5 and 1.8.6:

  ntalbott@joshua:~/tmp/issue_refined$ multiruby process_test.rb 

  VERSION = 1.8.4

  Loaded suite process_test
  Started
  "I'm going to die..."
  .
  Finished in 0.010974 seconds.

  1 tests, 1 assertions, 0 failures, 0 errors

  RESULT = 0

  VERSION = 1.8.5

  Loaded suite process_test
  Started
  "I'm going to die..."
  .
  Finished in 0.01149 seconds.

  1 tests, 1 assertions, 0 failures, 0 errors

  RESULT = 0

  VERSION = 1.8.6

  Loaded suite process_test
  Started
  "I'm going to die..."
  .
  Finished in 0.011605 seconds.

  1 tests, 1 assertions, 0 failures, 0 errors

  RESULT = 0

  TOTAL RESULT = 0 failures

I'm not sure what to tell you - it appears that for some reason your Ruby is broken. Can you try building Ruby from scratch on your Linux box and seeing if you get the same error?

I'm going to re-close this - if you can figure out more specifically what the issue is with your box, please re-open it and I'll investigate again.

----------------------------------------------------------------------

Comment By: Jeff Xiong (gigix)
Date: 2007-03-21 05:04

Message:
I wrote a new test case for this issue. It doesn't depend on
any other libraries. And it seems to me like at_exit
structure has some problem with it. With following code:

at_exit do
  p "I'm going to die..."
  exit false
end

the exit code should be non-zero. But in my environment
(Ruby 1.8.6 on Linux and Windows) the exit code is 0.

Please check out the second attachment I uploaded
(issue_refined.zip) and try it out.

----------------------------------------------------------------------

Comment By: Nathaniel Talbott (ntalbott)
Date: 2007-03-20 19:47

Message:
I'm closing this in lieu of a reproducible failure. Feel free to re-open it if/when you can provide one.

----------------------------------------------------------------------

Comment By: Nathaniel Talbott (ntalbott)
Date: 2007-03-19 13:33

Message:
Can you give me an example that runs independent of a Rails app? For instance, you must have something extra installed in your app, since assert_false is not a standard test/unit assertion.

In my testing, both in a Rails app, and an independent test I set up, I couldn't duplication the problem.

----------------------------------------------------------------------

Comment By: Jeff Xiong (gigix)
Date: 2007-03-16 21:56

Message:
I posted an attachment (issue.zip). You can unzip it to a
Rails app's root directory, then run process_test.rb.

----------------------------------------------------------------------

Comment By: Nathaniel Talbott (ntalbott)
Date: 2007-03-16 07:01

Message:
Can you post an example that demonstrates the problem?

----------------------------------------------------------------------

You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=1698&aid=9300&group_id=426