Rob Doug wrote:
>> One culprit could be mechanize.  Mechanize keeps a history of the pages
>> you've visited, and if you don't limit that length it will grow 
>> infinitely.
>>
>> Try setting the max history:
>>
>>   http://mechanize.rubyforge.org/mechanize/WWW/Mechanize.html#M000173
> 
> Thanks, but I've already try this... mem leak happens even when I use 
> "net/http" with Hpricot

You'll have to share more of your implementation then.  You deduced that
the cause wasn't threads, but you've only shared the threaded part of
your code.

Also, have you run your program with valgrind to verify that it is in
fact a memory leak?  It could just be objects sitting around waiting for GC.

-- 
Aaron Patterson
http://tenderlovemaking.com/