Issue #5333 has been updated by Yusuke Endoh.

Status changed from Open to Third Party's Issue

Hello,

Some people told me that the issue reproduces on os x.  Thanks!

@nagachika, @n0kada and I investigated this issue.  As a result,
I'm sad that I'd have to say this is third party's issue.


The following patch will show the mechanism of this issue.

diff --git a/app/models/competition.rb b/app/models/competition.rb
index 9c0ee55..04f1446 100644
--- a/app/models/competition.rb
+++ b/app/models/competition.rb
@@ -1,3 +1,4 @@
+p :load
 class Competition < ActiveRecord::Base
   has_many :news
 
@@ -7,6 +8,7 @@ class Competition < ActiveRecord::Base
 private
   def cannot_end_before_it_started
     unless starts_at.nil? or ends_at.nil?
+      p :call
       errors.add(:ends_at, "can't be prior to start date") if ends_at < starts_at
     end
   end


With it applied, rake spec prints the following output (on my Ubuntu):

  $ rake spec
  (in /home/mame/work/ruby/coverage-bug)
  :load

  Competition
    GET /competitions/:id
  :load
  :call
      should display basic information

  Competition
    validations
  :call
      should require ends_at >= starts_at

  Finished in 0.55506 seconds
  2 examples, 0 failures
  ["/home/mame/work/ruby/coverage-bug/app/models/competition.rb",
   [1, 1, 1, nil, 1, 1, nil, 1, 1, 2, 2, 2, nil, nil, nil]]


So we can see the events occur in the following order on Ubuntu:
  :load -> :load -> :call -> :call

According to @nagachika, the trace on os x is:
  :load -> :call -> :call -> :load

when factory_girl is used.  Otherwise:
  :load -> :call -> :load -> :call

Reloading the same file means clearing the coverage of the file
so far.  To measure correct coverage, the event order should be:
  :load -> :call -> :call


I'm not sure which library is ill-designed, but I suspect
active_support; it seems to load app/models/competition.rb
twice via const_missing, which also causes such an unstable
(platform-dependent) event order, I guess.  You can see this
by replacing "p :load" with "p *caller" in the above patch.

Is active_support developer seeing this ticket?
Or could anyone inform them?

Thanks,

-- 
Yusuke Endoh <mame / tsg.ne.jp>
----------------------------------------
Bug #5333: Coverage library giving wrong results
http://redmine.ruby-lang.org/issues/5333

Author: Christoph Olszowka
Status: Third Party's Issue
Priority: Normal
Assignee: Yusuke Endoh
Category: lib
Target version: 1.9.2
ruby -v: -


I'm the author of the simplecov gem, a wrapper on top of the Coverage library. You can find it on Github at https://github.com/colszowka/simplecov

I keep getting reports from users that code they have cleary tested is not reported as covered (you can find it a discussion at https://github.com/colszowka/simplecov/issues/60). This was very hard to trace back since no one was able to give me code samples so far. A couple of days ago, a user finally came up with a sample rails application where this problem was visible in conjunction with FactoryGirl. 

I tried this out using the Coverage library directly, and the result was the same: Creating the record directly reported the file as covered, doing the same with FactoryGirl reported a coverage of 0 for certain lines - even though they were executed.

I forked the application and added some instructions to the readme. You can find it at https://github.com/colszowka/coverage-bug

I verified this on the latest 1.9.2 release as well as 1.9.3-preview1:

ruby 1.9.3dev (2011-07-31 revision 32789) [x86_64-darwin11.0.0]
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin11.0.0]

I'm not sure why this happens in conjunction with FactoryGirl, but the code inside the if-statement is being executed in that case as well as I verified by adding a raise inside there and getting a correctly failing rspec suite.

Please let me know if you need any further information on this


-- 
http://redmine.ruby-lang.org