> > I would like to see if anyone has gone down this road and can weigh in
> > on these options.
>
> How difficult it your task? If you were able to reduce (heh) it to a > MapReduce problem, you could use something like Skynet or Starfish. > For even simpler forking, check out Ara Howard's threadify or my > forkify for simple parallel processing.

Yes, it fits a MapReduce problem but most MapReduce implementations I
came across seemed like overkill. I wasn't aware of Skynet or
Starfish--they look promising, thanks. The file interface of Starfish
may in fact be just what I'm looking for.

I'll check out threadify and forkify too.

Thanks again.
-- Mark.