On 21.04.2007 13:23, Wincent Colaiuta wrote:
> I'm trying to work out ways to reduce the memory use of one of my
> projects, but I don't know what methods are available to the Ruby
> programmer for profiling memory use and tracking down garbage
> collection problems.
> 
> The short version:
> 
> I have a project where processing a file can consume dozens of
> megabytes of memory; if I process many files in a single run then
> total memory usage can reach hundreds of megabytes or more than a gig.
> I would expect garbage collection to kick in along the way but it
> doesn't seem to be happening, memory usage grows and grows, and I
> don't know where to start zeroing in on the problem.
> 
> The long version:
> 
> I've written an object-oriented templating system[1] that incorporates
> a memoizing packrat parser. As each file is parsed the parser
> "memoizes" the partial results for speed. In a lengthy file the size
> of the memoizing cache can grow quite large (dozens of megabytes). But
> I would expect the entire contents of the memoization cache to get
> garbage collected when I move on to the next file; the cache itself
> has definitely fallen out of scope by that time. But garbage
> collection doesn't seem to be happen, as memory use grows linearly as
> I batch process input files.

I don't know what techniques exactly you employ but it seems indeed that 
the cached data is held on to longer than intended.

> As this is a largish, complicated project I don't even know where to
> begin to start investigating this. So really, I am looking for general
> information on techniques for measuring and exploring memory use and
> garbage collection in Ruby.

I am not sure, there might be an extension around.  Did you check with 
the RAA?

What I'd do here is a) code inspection b) create a finalizer with some 
debugging output for your cache instance and see whether you see the 
output.  If not, you know that your cache instance is still referred to.

Kind regards

	robert