On Mon, Feb 25, 2008 at 4:29 PM, pluskid <pluskid / gmail.com> wrote: > Hi, all! > > I'm having memory leak problem with my program, but can't find where's > the leak. > I have a program written in pure-Ruby. It implemented a Chinese word > segment > algorithm. I call `segment(text)' to get the result. But each time I > call `segment', > the memory usage of my program (Ruby) increased several handred K- > bytes -- the > number of bytes of increment is roughly equal during each call. > > I can see from outside (using `ps' or `top') that the memory is > leaking. But I can't > find where the leaking goes. I tried to use ObjectSpace.each_object to > find out > what leaks. I followed this article > > http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-with-rails > > and compare the objects after each call to `segment'. What surprise me > is that > there's no object leaked between each call. The number of objects > after each > call is roughly equal. > > I also tried bleak_house: > > http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html > > I don't precisely how to use it. I just dump a snapshot after each > call to `segment' > and use `bleak' to analyze the result. However, it also shows no leak. > But if > there's no leak, where goes the memory? > > Here's the result of using the `pmap' tool to inspect the running Ruby > process: > > 933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many > 08048000 4K r-x-- /usr/bin/ruby1.8 > 08049000 4K rw--- /usr/bin/ruby1.8 > 0804a000 33412K rw--- [ anon ] > b74a8000 7016K rw--- [ anon ] > b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so > b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so > b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/ > ruby_prof.so > b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/ > ruby_prof.so > b7b8c000 1192K rw--- [ anon ] > b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so > b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so > b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so > b7e00000 12K rw--- [ anon ] > b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so > b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so > b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so > b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so > b7e33000 156K rw--- [ anon ] > b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so > b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so > b7e5e000 4K rw--- [ anon ] > b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so > b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so > b7e75000 8K rw--- [ anon ] > b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6 > b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6 > b7f36000 64K rw--- [ anon ] > b7f48000 8K rw--- [ anon ] > b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so > b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so > b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so > b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so > b7f51000 8K rw--- [ anon ] > b7f53000 104K r-x-- /lib/ld-2.7.so > b7f6d000 8K rw--- /lib/ld-2.7.so > bfdea000 116K rw--- [ stack ] > ffffe000 4K r-x-- [ anon ] > total 44568K > > The section "0804a000 33412K rw--- [ anon ]" increased after each > call. I've no idea now. Do you have any suggestion for me? This is what I have read here or somewhere else, and I an not knowledgeable enough to confirm it, anyway: Ruby allocs space from the OS in large chunks (initially 8 MB, and each other chunk is bigger by a predefined coefficient). Any chunk can be returned to the OS only if it is empty - so if your memory is fragmented, ruby will have to alloc new chunks in order to satisfy ruby script being run. If this is true, it means: the order and size of your allocations is important. Especially order of the allocations that you later release relative to those that you don't.