sam.saffron / gmail.com wrote:
> Issue #13618 has been updated by sam.saffron (Sam Saffron).
> 
> > I'm leaning towards Thread::Green, so existing users can do 
> > s/Thread.new/Thread::Green.new/ in many cases.
> 
> Yes I think this works the problem though is that people will
> expect this to work like green threads, meaning they also 
> should auto-yield regularly. You should be allowed to have
> two green threads doing expensive computations. One tight loop
> in a reactor now and you blow up everything (unlike normal threads)

Good point.  rb_thread_call_without_gvl could be used to migrate
work to a thread pool (so we end up with M:N threads),
and maybe that's not horrible as a default behavior.
Data migration across native threads would hurt locality-wise
for short-lived tasks (e.g. rb_stat), though...

Fwiw, I was planning on adding a hinting mechanism to
rb_thread_call_without_gvl anyways later on (for GC, maybe);
but hints could be added to prevent/encourage migration based
on the expected duration/bottleneck of the function.

> This would mean you would have to pull in the 1.8 scheduler 
> or something. But then this stops being a proper reactor :(.
> 
> I guess this is the underlying reason you just wanted to call this auto
> yielding fibers instead of threads to start with. 

Right, the predictability of not having a timer switch threads
automatically is appealing, sometimes.

Having rb_thread_call_without_gvl become a scheduling point of
some sort for green threads would be fine, however, since all
callers already assume a context switch will happen.

> A question for Matz and Koichi is if they expect the scheduler from 1.8 
> to be brought back, if this is "safe" by default and "opt-in" for unsafe.
> 
> 
> > I expect PG to be able to benefit from rb_wait_for_single_fd when
> > using sockets. I know mysql2 uses rb_wait_for_single_fd, at least.
> 
> I am not sure about this, libpq abstracts all of this stuff away from you
> this is why Sean G wrote a complete binary protocol implementation in rust, 
> to gain control. pg gem does not use rb_wait_for_single_fd it just releases gvl. 
> 
> We have to make sure there is some sort of path forward with Postgres here
> it is a huge issue. 

I don't know how expensive it is to parse the Pg protocol;
but I remember in the 1.8 days pg was one of the few gems to
use rb_thread_select and it played nicely with 1.8 green threads.
Can't say I know Pg well these days, it's been over a decade
since I used it with Ruby.

> MiniRacer is CPU bound its basically packaging libv8 into
> Ruby. I am on the fence here On one hand it would be nice to
> auto yield so we feel reduced GVL pain and Ruby code can run
> while v8 does it's thing. On the other hand the semantics of
> one thread at 100% suddenly becomes 2 threads at 100% is not
> ideal. Hard to decide.  

How long does it release the GVL for?  The thread pool /
workqueue idea I mentioned above might be a good fit for this
if the communications overhead can masked by the length of the
task.  Nothing wrong with 2 threads at 100% if they're getting
work done faster than 1 thread at 100%.


I wouldn't want a native pool to be used for something like
getaddrinfo, however, that's hugely inefficient (but exactly
what getaddrinfo_a does internally in glibc).

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>