On Monday 07 December 2009 09:20:08 am Brian Candler wrote:
> David Masover wrote:
> > The question you should be asking is, "What's the best way to handle
> > concurrency in Ruby?" The answer is, it depends what you're doing, but
> > it's
> > probably not threads.
> 
> Ruby threads *are*
> often useful, especially when used in a coarse-grained way.

I agree. However, I don't think threads are the best primitive to use for 
coarse-grained multithreading. I much prefer processes and message-passing.

> For example, suppose you have a bunch of objects and each is opening a
> HTTP connection to some remote server and pulling down content, and you
> want this to happen concurrently. Each object is essentially
> self-contained. Having these doing concurrent downloads within threads
> is straightforward to program and pretty robust.

I agree, and this is how I do that -- I should clarify. I like threads 
technologically. I think they can be much cleaner than Unix processes. A 
fork() is nice to prevent one crash from bringing down your entire app -- but 
your app shouldn't be crashing that badly in the first place.

You mentioned Erlang. It will do some N:M threading -- that is, there really 
will be some OS threads involved. In theory, one crash could bring down your 
entire app. Also in theory, the Erlang runtime is robust enough that this will 
Never Happen -- and to ensure that, the preferred way to write C extensions is 
as separate processes which talk to Erlang via RPC. More efficient than fork 
on Unix, but much more reliable than "threads" in just about any language.

That is: I see threads as both as harmful and as useful as Goto. All CPUs 
essentially implement Goto, but no one in their right mind codes in terms of 
Goto. We abstract it away, and use structured code.

> The alternatives aren't pretty: rewrite your application in an
> event-driven way (so you have to find a HTTP client library which works
> this way too),

A quick Google turns up Rev::HttpClient, so this probably wasn't the best 
example.

> or fork off separate processes (which then have to
> communicate back to the central one with the results, which might mean
> select'ing across the children, or using the filesystem as a temporary
> data store)

Or abstracting this away until it's more manageable. You can do that with 
threads, too, but in Ruby, more processes means more concurrency, unless 
you're doing JRuby -- and it definitely means something safer.

> Of course, the assumption here is that you're programming in Ruby. If
> you want to avoid threads (which I agree is a good thing to do) and
> still have concurrency, then it might be better to switch to Erlang
> rather than jump through hoops in Ruby.

That's probably true, if you can manage it -- but even in Ruby, there are 
things that will abstract away threads for you.

The biggest problem I have with Erlang is that the syntax is hideous, 
especially after Ruby. The second biggest problem I have is that while it 
handles concurrency and binary data very well, Ruby handles just about 
everything else better -- Unicode, string processing, metaprogramming and 
reflection, DSLs...

This is why I have such high hopes for Reia, and why I'm tempted to dabble in 
io -- I want something that's at least as beautiful as Ruby (though I do like 
prototypal inheritance), but at least as good at concurrency as Erlang.

But in the mean time, I'm going to say that processes are likely to have way 
fewer surprises for the average newbie, while hypocritically building wrappers 
around threads for fun.