Issue #16786 has been updated by duerst (Martin D=FCrst).


ioquatix (Samuel Williams) wrote in #note-34:
> Thanks Matz.
> =

> > since the fiber created by the method is not the original fiber at all.
> =

> Can you clarify "not the original fiber at all"? It's the same way `Integ=
er(...)` creates instance of `class Integer`.

I can't speak for Matz, but my guess is that he meant "not the original typ=
e of fiber", i.e. not the same as you'd get e.g. with `Fiber.new`.

----------------------------------------
Feature #16786: Light-weight scheduler for improved concurrency.
https://bugs.ruby-lang.org/issues/16786#change-85616

* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
----------------------------------------
# Abstract

We propose to introduce a light weight fiber scheduler, to improve the conc=
urrency of Ruby code with minimal changes.

# Background

We have been discussing and considering options to improve Ruby scalability=
 for several years. More context can be provided by the following discussio=
ns:

- https://bugs.ruby-lang.org/issues/14736
- https://bugs.ruby-lang.org/issues/13618

The final Ruby Concurrency report provides some background on the various i=
ssues considered in the latest iteration: https://www.codeotaku.com/journal=
/2020-04/ruby-concurrency-final-report/index

# Proposal

We propose to introduce the following concepts:

- A `Scheduler` interface which provides hooks for user-supplied event loop=
s.
- Non-blocking `Fiber` which can invoke the scheduler when it would otherwi=
se block.

## Scheduler

The per-thread fiber scheduler interface is used to intercept blocking oper=
ations. A typical implementation would be a wrapper for a gem like EventMac=
hine or Async. This design provides separation of concerns between the even=
t loop implementation and application code. It also allows for layered sche=
dulers which can perform instrumentation, enforce constraints (e.g. during =
testing) and provide additional logging. You can see a [sample implementati=
on here](https://github.com/socketry/async/pull/56).

```ruby
class Scheduler
  # Wait for the given file descriptor to become readable.
  def wait_readable(io)
  end

  # Wait for the given file descriptor to become writable.
  def wait_writable(io)
  end

  # Wait for the given file descriptor to match the specified events within
  # the specified timeout.
  # @param event [Integer] a bit mask of +IO::WAIT_READABLE+,
  #   `IO::WAIT_WRITABLE` and `IO::WAIT_PRIORITY`.
  # @param timeout [#to_f] the amount of time to wait for the event.
  def wait_any(io, events, timeout)
  end

  # Sleep the current task for the specified duration, or forever if not
  # specified.
  # @param duration [#to_f] the amount of time to sleep.
  def wait_sleep(duration =3D nil)
  end

  # The Ruby virtual machine is going to enter a system level blocking
  # operation.
  def enter_blocking_region
  end

  # The Ruby virtual machine has completed the system level blocking
  # operation.
  def exit_blocking_region
  end

  # Intercept the creation of a non-blocking fiber.
  def fiber(&block)
    Fiber.new(blocking: false, &block)
  end

  # Invoked when the thread exits.
  def run
    # Implement event loop here.
  end
end
```

A thread has a non-blocking fiber scheduler. All blocking operations on non=
-blocking fibers are hooked by the scheduler and the scheduler can switch t=
o another fiber. If any mutex is acquired by a fiber, then a scheduler is n=
ot called; the same behaviour as blocking Fiber.

Schedulers can be written in Ruby. This is a desirable property as it allow=
s them to be used in different implementations of Ruby easily.

To enable non-blocking fiber switching on blocking operations:

- Specify a scheduler: `Thread.current.scheduler =3D Scheduler.new`.
- Create several non-blocking fibers: `Fiber.new(blocking:false) {...}`.
- As the main fiber exits, `Thread.current.scheduler.run` is invoked which
  begins executing the event loop until all fibers are finished.

### Time/Duration Arguments

Tony Arcieri suggested against using floating point values for time/duratio=
ns, because they can accumulate rounding errors and other issues. He has a =
wealth of experience in this area so his advice should be considered carefu=
lly. However, I have yet to see these issues happen in an event loop. That =
being said, round tripping between `struct timeval` and `double`/`VALUE` se=
ems a bit inefficient. One option is to have an opaque argument that respon=
ds to `to_f` as well as potentially `seconds` and `microseconds` or some ot=
her such interface (could be opaque argument supported by `IO.select` for e=
xample).

### File Descriptor Arguments

Because of the public C interface we may need to support a specific set of =
wrappers for CRuby.

```c
int rb_io_wait_readable(int);
int rb_io_wait_writable(int);
int rb_wait_for_single_fd(int fd, int events, struct timeval *tv);
```

One option is to introduce hooks specific to CRuby:

```ruby
class Scheduler
  # Wrapper for rb_io_wait_readable(int) C function.
  def wait_readable_fd(fd)
    wait_readable(::IO.from_fd(fd, autoclose: false))
  end

  # Wrapper for rb_io_wait_readable(int) C function.
  def wait_writable_fd(fd)
    wait_writable(::IO.from_fd(fd, autoclose: false))
  end

  # Wrapper for rb_wait_for_single_fd(int) C function.
  def wait_for_single_fd(fd, events, duration)
    wait_any(::IO.from_fd(fd, autoclose: false), events, duration)
  end
end
```

Alternatively, in CRuby, it may be possible to map from `fd` -> `IO` instan=
ce. Most C schedulers only care about file descriptor, so such a mapping wi=
ll introduce a small performance penalty. In addition, most C level schedul=
ers will not care about `IO` instance.

## Non-blocking Fiber

We propose to introduce per-fiber flag `blocking: true/false`.

A fiber created by `Fiber.new(blocking: true)` (the default `Fiber.new`) be=
comes a "blocking Fiber" and has no changes from current Fiber implementati=
on. This includes the root fiber.

A fiber created by `Fiber.new(blocking: false)` becomes a "non-blocking Fib=
er" and it will be scheduled by the per-thread scheduler when the blocking =
operations (blocking I/O, sleep, and so on) occurs.

```ruby
Fiber.new(blocking: false) do
  puts Fiber.current.blocking? # false

  # May invoke `Thread.scheduler&.wait_readable`.
  io.read(...)

  # May invoke `Thread.scheduler&.wait_writable`.
  io.write(...)

  # Will invoke `Thread.scheduler&.wait_sleep`.
  sleep(n)
end.resume
```

Non-blocking fibers also supports `Fiber#resume`, `Fiber#transfer` and `Fib=
er.yield` which are necessary to create a scheduler.

### Fiber Method

We also introduce a new method which simplifes the creation of these non-bl=
ocking fibers:

```ruby
Fiber do
  puts Fiber.current.blocking? # false
end
```

This method invokes `Scheduler#fiber(...)`. The purpose of this method is t=
o allow the scheduler to internally decide the policy for when to start the=
 fiber, and whether to use symmetric or asymmetric fibers.

If no scheduler is specified, it is a error: `RuntimeError.new("No schedule=
r is available")`.

In the future we may expand this to support some kind of default scheduler.

## Non-blocking I/O

`IO#nonblock` is an existing interface to control whether I/O uses blocking=
 or non-blocking system calls. We can take advantage of this:

- `IO#nonblock =3D false` prevents that particular IO from utilising the sc=
heduler. This should be the default for `stderr`.
- `IO#nonblock =3D true` enables that particular IO to utilise the schedule=
r. We should enable this where possible.

As proposed by Eric Wong, we believe that making I/O non-blocking by defaul=
t is the right approach. We have expanded his work in the current implement=
ation. By doing this, when the user writes `Fiber do ... end` they are guar=
anteed the best possible concurrency possible, without any further changes =
to code. As an example, one of the tests shows `Net::HTTP.get` being used i=
n this way with no further modifications required.

To support this further, consider the counterpoint, that `Net::HTTP.get(...=
, blocking: false)` is required for concurrent requests. Library code may n=
ot expose the relevant options, sevearly limiting the user's ability to imp=
rove concurrency, even if that is what they desire.

# Implementation

We have an evolving implementation here: https://github.com/ruby/ruby/pull/=
3032 which we will continue to update as the proposal changes.

# Evaluation

This proposal provides the hooks for scheduling fibers. With regards to per=
formance, there are several things to consider:

- The impact of the scheduler design on non-concurrent workloads. We believ=
e it's acceptable.
- The impact of the scheduler design on concurrent workloads. Our results a=
re promising.
- The impact of different event loops on throughput and latency. We have in=
dependent tests which confirm the scalability of the approach.

We can control for the first two in this proposal, and depending on the des=
ign we may help or hinder the wrapper implementation.

In the tests, we provide a basic implementation using `IO.select`. As this =
proposal is finalised, we will introduce some basic benchmarks using this a=
pproach.

# Discussion

The following points are good ones for discussion:

- Handling of file descriptors vs `IO` instances.
- Handling of time/duration arguments.
- General design and naming conventions.
- Potential platform issues (e.g. CRuby vs JRuby vs TruffleRuby, etc).

The following is planned to be described by @eregon in another design docum=
ent:

- Semantics of non-blocking mutex (e.g. `Mutex.new(blocking: false)` or som=
e other approach).

In the future we hope to extend the scheduler to handle other blocking oper=
ations, including name resolution, file I/O (by `io_uring`) and others. We =
may need to introduce additional hooks. If these hooks are not defined on t=
he scheduler implementation, we will revert back to the blocking implementa=
tion where possible.



-- =

https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=3Dunsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>