Issue #10682 has been updated by Charles Nutter.


Patch updated: https://gist.github.com/headius/4226cd94bbcf7b150e65

I only made the rescue change requested by nobu. If there's a strong preference to use prepend or to add "exclude" once as a singleton method to some superclass, I can try it...but I'm happy with the way this works right now.

Some of the complexity is also due to some sub-suites using test/unit, so I'm not sure there's a single place we could define exclude...it would probably need to live in both test/lib/minitest and test/lib/test/unit.

Please let me know any remaining objections/concerns/suggestions by Monday, so I can commit this.

I will try to look into the leakchecker and tracepointchecker issues on JRuby, and file issues to improve them if the issues are because of implementation-specific behavior.

----------------------------------------
Feature #10682: Add "excludes" support to test suite, for alternative implementations and platforms
https://bugs.ruby-lang.org/issues/10682#change-50760

* Author: Charles Nutter
* Status: Open
* Priority: Normal
* Assignee: 
* Category: test
* Target version: current: 2.2.0
----------------------------------------
JRuby uses MRI's test suite as our primary compatibility suite. We would like to enhance the suite to support excluding tests.

Before the juggling of minitest versions in stdlib, JRuby was using minitest-excludes to exclude tests we knew didn't pass. The EXCLUDE_DIR env pointed at a dir with .rb files named after the test classes and containing "exclude" lines as seen here: https://github.com/jruby/jruby/blob/jruby-1_7/test/externals/ruby1.9/excludes/TestBignum.rb

This allowed us to maintain a "high water mark" for compatibility as MRI's suite evolved. It is largely the same feature as mspec/RubySpec's "tags".

Unfortunately minitest 5 does not support excludes, and MRI's suite has frozen a version of minitest 4 to use that is incompatible with the minitest-excludes library. In addition, many tests have returned to running with test/unit.

I provide here a patch for the exclude support we added to our copy of MRI's suite: https://gist.github.com/headius/4226cd94bbcf7b150e65

The only significant changes from minitest-excludes:

* The env var is now "EXCLUDES". This is negotiable, but I don't know if enough people are using (or ever will use again) minitest-excludes, so the compatibility issue is moot.
* There's no require needed to activate the excludes; if the env var is present, it will attempt to load them.

I'm willing to tune and improve this as necessary. JRuby master (9k, 9.0.0.0, 2.2-compatible) has been running MRI's tests this way for several months.



-- 
https://bugs.ruby-lang.org/