Issue #14183 has been updated by jeremyevans0 (Jeremy Evans).


mame,

Thanks for your continued work on this.

I still agree that for methods that accept keyword arguments, we should make changes to avoid the problems that currently exist for keyword arguments.

I still believe that breaking all backwards compatibility for methods that do not currently accept keyword arguments, just to allow keyword arguments to be added safely in the future, is not a worthy tradeoff, for the following reasons:

* Many if not most of the methods may never be converted to keyword arguments, in which case backwards compatibility is broken for no benefit.

* This encourages the use of keyword arguments, while the use of keyword arguments hurts performance in all cases where keyword splats are used (either at the caller side or the callee side).  The option hash approach can be made faster and allocation-less, while all keyword splats are currently slower as they require allocations.  I'm not sure that the keyword argument splat performance issues could be fixed without breaking backwards compatibility for all keyword argument splats.

* For methods that currently use option hashes, requiring braces around the option hash can make it more difficult to convert to keyword arguments, not less.  A method such as `def foo(opts={}) end` that is usually called using `foo(bar: 1)`, will still work if you switch to keyword arguments: `def foo(**opts) end`.  It is true that the `foo(hash)` calling style would require modifications with the switch to keyword arguments, though.

I think the biggest problem with keeping backwards compatibility for methods that do not accept keyword arguments is handling delegation.

~~~ruby
def foo(*a, **kw, &block)
  bar(*a, **kw, &block)
end
~~~

I believe with your proposal, this is expected to work regardless of whether `bar` accepts keyword arguments.  If `bar` doesn't accept keyword arguments, then calling foo with a keyword argument will raise an exception when `foo` calls `bar`.  I think one possible way to get that simple delegation to work would be to allow double-splat when calling methods that do not accept keyword arguments (keep backwards compatibility).  For example, allow this:

~~~ruby
def bar(hash={})
  hash[:a]
end

bar(**{a: 1})
# => 1

foo(**{a: 1})
# => 1
~~~

This keeps backwards compatibility back to Ruby 2.0.  It will also make it easier to transition such code to keyword arguments later without breaking backwards compatibility, since changing the definition of `bar` to `def bar(**hash) hash[:a] end` would still work in that case.

The main problematic case would be if `bar` accepted a positional splat but did not accept keyword arguments, where an empty hash would be provided if no keyword arguments were used:

~~~ruby
def bar(*a)
  a
end

bar
# => []

foo
# => [{}]
~~~

One possible way around that would be that if a method accepts a positional splat and does not accept keyword arguments, then calling the method with an empty keyword argument splat would not pass a positional argument.  Proposed behavior:

~~~ruby
def bar(*a)
  a
end

bar
# => []

foo
# => []

bar(**{})
# => []

foo(**{})
# => []

bar(1, a: 1)
# => [1, {a: 1}]

foo(1, a: 1)
# => [1, {a: 1}]
~~~

# My Proposed Alternative

To sum up, here is my proposed alternative approach:

* For methods that accept keyword arguments, the same as your proposal
* For methods that do not accept keyword arguments:
    * Allow braceless hashes as positional arguments (keep backwards compatibility)
    * Allow **keyword splats
        * If keyword is empty hash, do not add the empty hash positional argument (new behavior)
        * Otherwise, add keyword as positional hash argument (keep backwards compatibility)


I think this alternative proposal handles "2. Explicit Delegation of keywords backfires" and "3. There are many unintuitive corner cases".  It does not handle "1. Keyword extension is not always safe".  However, I believe you could keep safe keyword extension if using keyword splat, using an approach that works and is backwards compatible to Ruby 2.0.  From your example:

~~~ruby
# Before
def foo(*args)
  p args
end
foo(key: 42)
# => [{:key=>42}]

# Add keyword arguments
def foo(*args, output: $stdout, **kw)
  args << kw
  output.puts args.inspect
end
foo(key: 42)
# => [{:key=>42}]
~~~

# Issues with keyword-argument-separation branch

In terms of the specific implementation in your keyword-argument-separation branch:

The rb_no_keyword_hash approach breaks modification of the hash, which I believe is unexpected:

~~~ruby
def foo(**opts)
  opts
end

foo
#  => {(NO KEYWORD)}

def foo(**opts)
  opts[:a] = 1
  opts
end

foo
# FrozenError (can't modify frozen Hash)
~~~

It may be possible to work around that by setting a flag on the hash instead of using a shared frozen hash, assuming there is a spare flag we can use for that purpose.  If a flag isn't available, we probably could use an instance variable that doesn't start with `@` (making it only visible to C).

The warning seems inconsistent.  For positional splats, you get warned if the braceless hash is the first argument, but not if it is a subsequent argument:

~~~ruby
def bar(*a)
  a
end

bar
=> []

bar(a: 1)
# warning: The keyword argument for `bar' (defined at XXX) is used as the last parameter
# => [{:a=>1}]

bar(1, a: 1)
# => [1, {:a=>1}]
~~~

This situation also occurs for methods without splats where both arguments are optional (and maybe other cases):

~~~ruby
def baz(a=1, b={})
  [a, b]
end

baz
# => [1, {}]

baz(a: 2)
# warning: The keyword argument for `baz' (defined at XXX) is used as the last parameter
[{:a=>2}, {}]

baz(1, a: 2)
# => [1, {:a=>2}]
~~~

Is that behavior in regards to warnings expected?

Behavior is different for methods defined in C, as C methods are always passed a hash, so the brace, braceless, and splat forms all work:

~~~ ruby
String.new(capacity: 1000)
# => ""
String.new({capacity: 1000})
# => ""
String.new(**{capacity: 1000})
# => ""
~~~

This results in inconsistent behavior depending how how the method is defined.  This will lead to backwards compatibility problems if you move a method definition from C to ruby, or if you have a method defined in both C and ruby, with the pure ruby version used as a fallback if the C version cannot be used.

I look forward to discussing this issue in person at the developer meeting next month.

----------------------------------------
Feature #14183: "Real" keyword argument
https://bugs.ruby-lang.org/issues/14183#change-77141

* Author: mame (Yusuke Endoh)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: Next Major
----------------------------------------
In RubyWorld Conference 2017 and RubyConf 2017, Matz officially said that Ruby 3.0 will have "real" keyword arguments.  AFAIK there is no ticket about it, so I'm creating this (based on my understanding).

In Ruby 2, the keyword argument is a normal argument that is a Hash object (whose keys are all symbols) and is passed as the last argument.  This design is chosen because of compatibility, but it is fairly complex, and has been a source of many corner cases where the behavior is not intuitive.  (Some related tickets: #8040, #8316, #9898, #10856, #11236, #11967, #12104, #12717, #12821, #13336, #13647, #14130)

In Ruby 3, a keyword argument will be completely separated from normal arguments.  (Like a block parameter that is also completely separated from normal arguments.)
This change will break compatibility; if you want to pass or accept keyword argument, you always need to use bare `sym: val` or double-splat `**` syntax:

```
# The following calls pass keyword arguments
foo(..., key: val)
foo(..., **hsh)
foo(..., key: val, **hsh)

# The following calls pass **normal** arguments
foo(..., {key: val})
foo(..., hsh)
foo(..., {key: val, **hsh})

# The following method definitions accept keyword argument
def foo(..., key: val)
end
def foo(..., **hsh)
end

# The following method definitions accept **normal** argument
def foo(..., hsh)
end
```

In other words, the following programs WILL NOT work:

```
# This will cause an ArgumentError because the method foo does not accept keyword argument
def foo(a, b, c, hsh)
  p hsh[:key]
end
foo(1, 2, 3, key: 42)

# The following will work; you need to use keyword rest operator explicitly
def foo(a, b, c, **hsh)
  p hsh[:key]
end
foo(1, 2, 3, key: 42)

# This will cause an ArgumentError because the method call does not pass keyword argument
def foo(a, b, c, key: 1)
end
h = {key: 42}
foo(1, 2, 3, h)

# The following will work; you need to use keyword rest operator explicitly
def foo(a, b, c, key: 1)
end
h = {key: 42}
foo(1, 2, 3, **h)
```

I think here is a transition path:

* Ruby 2.6 (or 2.7?) will output a warning when a normal argument is interpreted as keyword argument, or vice versa.
* Ruby 3.0 will use the new semantics.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>