Issue #14723 has been updated by ko1 (Koichi Sasada).


My concerns are:

(1) Full GC (like GC.start) or step incremental marking/sweeping (to guarantee (or to reduce) the worst stopping time because of GC for every IO operation).
(2) How to know GC is required (if we invoke GC.start on any I/O (blocking) operations, it should be harmful)

Seeing your comment #2,

> Therefore, if GVL is uncontended and GC has work to-do, we use
> zero-timeout ppoll and do incremental GC work (incremental mark
> + lazy sweep) as long as we need to wait on FD.

they should be:

(1) do a step for incremental marking/sweeping
(2) only when incremental marking or sweeping

They are very reasonable for me.

My understanding, your proposal in pseudo code is (pls correct me if it is wrong):

```
def io_operation  
  while true
    if !GVL.contended? && GC.has_incremental_task?
      if result = io_operation(timeout: 0) > 0 # timeout = 0 means return immediately
        # There are result
        return result
       else
        GC.do_step
       end
    else
      GVL.release
      io_operation(timeout = long time)
      GVL.acquire
    end
  end
end
```

No problem for me.

However, your code https://80x24.org/spew/20180429035007.6499-3-e / 80x24.org/raw

```
+int
+rb_gc_step(const rb_execution_context_t *ec)
+{
+    rb_objspace_t *objspace = rb_ec_vm_ptr(ec)->objspace;
+
+    gc_rest(objspace);
+
+    return rb_gc_inprogress(ec);
+}
```

`gc_rest()` do all of rest steps.  Is it intentional?


Another tiny comments:

> +    static const struct timespec zero;

`zero` doesn't seem to be initialized. intentional?


----

Note:

After introducing Guild, getting `contended` status should be high-cost (we need to use lock to see this info).
However, we can eliminate this check if we shrink the target: only have one Guild (== current MRI).


----------------------------------------
Feature #14723: [WIP] sleepy GC
https://bugs.ruby-lang.org/issues/14723#change-71746

* Author: normalperson (Eric Wong)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
The idea is to use "idle time" when process is otherwise sleeping
and using no CPU time to perform GC.  It makes sense because real
world traffic sees idle time due to network latency and waiting
for user input.

Right now, it's Linux-only.  Future patches will affect other sleeping
functions:

  IO.select, Kernel#sleep, Thread#join, Process.waitpid, etc...

I don't know if this patch can be implemented for win32, right
now it's just dummy functions and that will be somebody elses
job.  But all pthreads platforms should eventually benefit.


Before this patch, the entropy-dependent script below takes 95MB
consistently on my system.  Now, depending on the amount of
entropy on my system, it takes anywhere from 43MB to 75MB.

I'm using /dev/urandom to simulate real-world network latency
variations.  There is no improvement when using /dev/zero
because the process is never idle.

  require 'net/http'
  require 'digest/md5'
  Thread.abort_on_exception = true
  s = TCPServer.new('127.0.0.1', 0)
  len = 1024 * 1024 * 1024
  th = Thread.new do
    c = s.accept
    c.readpartial(16384)
    c.write("HTTP/1.0 200 OK\r\nContent-Length: #{len}\r\n\r\n")
    IO.copy_stream('/dev/urandom', c, len)
    c.close
  end

  addr = s.addr
  Net::HTTP.start(addr[3], addr[1]) do |http|
    http.request_get('/') do |res|
      dig = Digest::MD5.new
      res.read_body { |buf|
        dig.update(buf)
      }
      puts dig.hexdigest
    end
  end

The above script is also dependent on net/protocol using
read_nonblock.  Ordinary IO objects will need IO#nonblock=true
to see benefits (because they never hit rb_wait_for_single_fd)

* gc.c (rb_gc_inprogress): new function
  (rb_gc_step): ditto
* internal.h: declare prototypes for new gc.c functions
* thread_pthread.c (gvl_contended_p): new function
* thread_win32.c (gvl_contended_p): ditto (dummy)
* thread.c (rb_wait_for_single_fd w/ ppoll):
  use new functions to perform GC while GVL is uncontended
  and GC is lazy sweeping or incremental marking
  [ruby-core:86265]
```

2 part patch broken out
https://80x24.org/spew/20180429035007.6499-2-e / 80x24.org/raw
https://80x24.org/spew/20180429035007.6499-3-e / 80x24.org/raw

Also on my "sleepy-gc" git branch @ git://80x24.org/ruby.git


---Files--------------------------------
sleepy-gc-wip-v1.diff (5.37 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>