Issue #16936 has been updated by jaruga (Jun Aruga).


https://github.com/ruby/ruby/blob/465b5dc124917b828a5964c50c4e470a0c03dcf4/tool/lib/test/unit.rb#L742

The following part matches only `type` (= `test_methods`) of the suites (`Array` of testing class such as `TestBugReporter`) It seems the testing class in `filter` is not considered.

```
        total = if filter
                  suites.inject(0) {|n, suite| n + suite.send(type).grep(filter).size}
                else
                  suites.inject(0) {|n, suite| n + suite.send(type).size}
                end
```

However if it matches only testing method, why the following command with the positive PATTERN matches the 2 test methods as expected? Actually I think `0` in `[1/0]` (`tool/lib/test/unit.rb`#_prepare_run` `total` = `0`) is wrong number in the case. That should be `2`.

```
$ make test-all TESTS="-v -n /\^TestBugReporter#test_bug_reporter_add\$$/ -n /\^TestProcess#test_status_quit\$$/"
...
[1/0] TestBugReporter#test_bug_reporter_add = 0.41 s
[2/0] TestProcess#test_status_quit = 0.35 s
Finished tests in 9.392046s, 0.2129 tests/s, 1.5971 assertions/s.
2 tests, 15 assertions, 0 failures, 0 errors, 0 skips
...
```

Why the 2 tests are executed?
Possibly when printing and running the test, the `TestClass#test_method` is considered in the following part.

https://github.com/ruby/ruby/blob/465b5dc124917b828a5964c50c4e470a0c03dcf4/tool/lib/test/unit.rb#L833

```
        def print(s)
          case s
          when /\A(.*\#.*) = \z/ # <= Testing class is considered to run it.
            runner.new_test($1)
...


I want to see the testing class is supported for the pattern matching.

----------------------------------------
Bug #16936: `make test-all TESTS="-n !/Foo#method/"` not skipping the test case
https://bugs.ruby-lang.org/issues/16936#change-86023

* Author: jaruga (Jun Aruga)
* Status: Closed
* Priority: Normal
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
On the current latest master `cf1adf985ab78507250db0e217a0fdd779e2c6e6`.

```
$ autoconf

$ ./configure --prefix=$(pwd)/dest

$ make
```

The following `make test-all` works as I mentioned at #16935.

```
$ make test-all TESTS="-v -n /\^TestBugReporter#test_bug_reporter_add\$$/ -n /\^TestProcess#test_status_quit\$$/"
...
[1/0] TestBugReporter#test_bug_reporter_add = 0.41 s
[2/0] TestProcess#test_status_quit = 0.35 s
Finished tests in 9.392046s, 0.2129 tests/s, 1.5971 assertions/s.
2 tests, 15 assertions, 0 failures, 0 errors, 0 skips
...
```

But it seems that the following `TestBugReporter#test_bug_reporter_add` and `TestProcess#test_status_quit` are not actually skipped.
Is it a bug?

```
$ make test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/\^TestBugReporter#test_bug_reporter_add\$$/ -n \!/\^TestProcess#test_status_quit\$$/" 2>&1 | tee make.log
Run options:-
  --seed=21367
  "--ruby=./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems"
  --excludes-dir=./test/excludes
  --name=!/memory_leak/
  -v  
  -n  
  "!/^TestBugReporter#test_bug_reporter_add$/"
  -n  
  "!/^TestProcess#test_status_quit$/"

# Running tests:

[  1/142] TestBugReporter#test_bug_reporter_add = 0.49 s
...
[128/142] TestProcess#test_status_quit = 0.30 s
...
Finished tests in 25.978847s, 5.4660 tests/s, 39.2627 assertions/s.
142 tests, 1020 assertions, 0 failures, 0 errors, 0 skips
```

I want to skip the specific test cases by `make test-all TESTS="..."` like this. I want to specify both the testing class (ex. `TestBugReporter`) and method (ex. `test_bug_reporter_add`) with the regular expression perfect matching to avoid unintended test cases are skipped.

```
$ make test-all TESTS="-v -n \!/\^TestBugReporter#test_bug_reporter_add\$$/ -n \!/\^TestProcess#test_status_quit\$$/"
```

The following `make test` works skipping `test_bug_reporter_add` and `test_status_quit` methods.

```
$ make test-all TESTS="test/-ext-/bug_reporter/test_bug_reporter.rb test/ruby/test_process.rb -v -n \!/test_bug_reporter_add\$$/ -n \!/test_status_quit\$$/" 2>&1 | tee make2.log
...
140 tests, 1005 assertions, 0 failures, 0 errors, 0 skips
```




-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>