2006/5/5, Don Stocks <amillionhitpoints / yahoo.com>:
> I am just learning Ruby and planning a to use it for a network
> monitoring system.  I would like to design a system that will scale well
> and handle large work loads ( tens of thousands of monitors per server
> ). My question is about concurrency. I am looking at threads and have
> fiddled with them a bit.  Ruby makes threading pretty straight forward.
> Now I'm trying to figure how to best use them.  The design goal is to
> have a server host tens of thousands of monitors in the most efficient
> way.  Monitors will need to be executed at specific and varying
> intervals.  (i.e. 1000 monitors once a minute / 2000 monitors once every
> three minutes / 5000 monitors once every 5 minutes / 10,000 monitors
> once every 10 minutes, etc.)
>
> This is a problem that I'm sure has already been solved.  I would
> appreciate any suggestions that the group can offer regarding a proven
> design to tackle this goal.

Hi Don,

with these amount of tasks it's inefficient to have a thread per
monitor. Things like this are typically tackled with a light weight
processing framework; you have a pool of threads that get feed tasks
via a thread safe queue (tomcat does it's request processing similarly
although with a bit more whistles and bells). Ruby comes with
everything you need for that apart from a scheduler maybe (but better
check the RAA). You then "just" have to glue pieces together.  I found
Doug Lea's book very valuable (although it's written for Java the
basic principles and strategies are the same):

http://www.amazon.com/gp/product/0201310090
http://g.oswego.edu/

HTH

Kind regards

robert

--
Have a look: http://www.flickr.com/photos/fussel-foto/