Currently, ruby/spec is maintained mostly by individuals and enjoys the
contribution of a few CRuby committers.
Unfortunately, the task of making specs for all the new features and
changes in MRI is a huge task and too much for these people.
If you look at specs for changes in Ruby 2.3, they were finally completed
around 1 year after 2.3.0 was out. This is too long and far from ideal.

I believe now is the time to change that.
If CRuby wants to define a language and allow alternative implementations
to target the Ruby language,
then contributing new specs for new features and changes to ruby/spec is
the best way they can help.

Currently, CRuby committers add tests under test/ or test/ruby/.
This is very hard to reuse for other implementations,
and more importantly does not document what the feature is supposed to do.
Instead, it provides assertions on specific data points.
This means, every time somebody read these tests, they have to reverse
engineer these assertions to understand the behavior and/or read MRI source
code.
Both of these tasks are taking a large amount of time and there is no point
for many people to have to do these tasks over and over again when the
behavior could be documented once clearly.

This is the strongest point for ruby/spec.
Every spec example ("test") has a description, that says how some
functionality is supposed to behave.
If these descriptions are done properly, there is no need to look at MRI
source or to reverse engineer the assertions to understand how the
functionality behaves.
Having this description, it is much faster to implement the correct
behavior, and test it.
Spec examples are also typically short and only contain a couple
assertions, each identifying a part of the behavior.
This is immensely useful for other implementations trying to implement a
new feature,
as they can implement the most important parts first, and come back to the
tricky edge cases later (and there are many in Ruby).
Also, they can update specs and then work on new specs one by one while
still running all previous specs.
No alternative Ruby implementation to this day passes all of ruby/spec,
neither MRI tests.
The ability to keep tests green while tagging (marking) known failures is
critical. This is built-in in ruby/spec.

There are many other technical considerations,
but in my opinion these descriptions and focused small spec examples are
the most important.
For instance, should vs expect().to() vs assert_equal is just syntactic
sugar and does not matter at all in comparison.

I will just give one example I know well, tests for File.join:
MRI tests:
https://github.com/ruby/ruby/blob/42d9712e78744e19ac71a7c0eaa099ac9203e1b1/test/ruby/test_file_exhaustive.rb#L1224-L1261
ruby/spec:
https://github.com/ruby/spec/blob/2a62f378c0c3151aa5abfe66a8eae2016b29c78a/core/file/join_spec.rb
Feel free to take another example. I only want to illustrate what I mean
about descriptions and small examples.

A point of friction which has been observed multiple times is that
ruby/spec is a different repository and there are synchronization problems.
For example, if a change happens on MRI trunk and it needs to adapt
ruby/spec, it will pass either ruby/ruby CI or ruby/spec CI but never both.
I want to fix this and I want to encourage CRuby developers to write specs.

Therefore I would like to integrate an in-tree copy of ruby/spec under
spec/rubyspec/.
This way, implementation and tests can be changed in a single commit, just
like with MRI tests.
I will keep synchronizing the changes to ruby/spec, so it can be used
directly and other people can contribute there.
I know this synchronization works because I have been doing it for over 2
years.
This process is also well documented:
https://github.com/ruby/spec/wiki/Merging-specs-from-JRuby-and-other-sources

There are many advantages if the person implementing the feature also
document the behavior:
The behavior will be documented by the most knowledgable person on the
subject.
It will also likely contain details about edge-cases and implementation,
which are very desirable for ruby/spec.
Descriptions are written in english, which might be a difficult for some
people less fluent in english.
But we are not writing literature, very simple english talking about Ruby
concepts and implementation is fine!

# So here is my request to the CRuby developers:
I would like to ask the CRuby developers to add specs in ruby/spec when
implementing new features, changing existing Ruby behavior and writing new
tests.
Of course, nobody likes to write tests twice, so I also propose these specs
can be done *instead* of MRI tests.
In my opinion, MRI tests should only be needed for MRI-specific behavior
such as RubyVM.
Regression tests are perfectly fine in ruby/spec.
If one Ruby implementation regresses or fail in some edge case, it's very
likely another will as well.

ruby/spec and mspec (the runner for ruby/spec) are not prefect and are not
fixed in stone, they can evolve and improve.
If you think mspec is too complex, we can simplify it.
If there is some feature in the MRI test harness not present in mspec, we
can add it.
If you think some syntax is confusing, let's discuss it and change it.

Dear CRuby developers, what do you think?
What are your concerns with this approach?
(supressed text/html)
Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>