Blackie <mudsweatandears / gmail.com> writes:

> If anyone can explain this I would appreciate it.

It's your OS (I meant kernel and libc).

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/160726
and
http://www.crowdedweb.com/articles/2006/03/22/ruby-on-rails-and-application-memory-consumption-patterns.

The second link is dead, but there may be cached copies around.

The first one shows that on some OS, if there is a hole in your memory
usage, that hole will not be returned by your libc to the kernel or
the kernel won't reclaim it. Either way, your OS is not taking it
back. 

The second one (if you ever find a copy) adds some more observations
and also a handy tweak for FreeBSD that cause the OS to take back
holes (IIRC).

But most importantly, one should know that RSS and VSZ are not
Accurate Measure of Memory Usage.

YS.



>
> I have an IT group knocking Ruby saying that it never releases memory
> back to the system (to be available to procs other than Ruby) so
> feeling somewhat defensive I went and wrote a dumb script that goes
> back and forth between two methods, one cat'ing huge strings and the
> other parsing an xml doc with Hpricot.
>
> "task" tops off (in top) around 50M and "other_task" peaks around 150M
> (on my machine, CentOS, lastest stable 1.8.6) but when we return to
> running "task" for extended periods, memory usage remains at ~150M.
>
> Forgive my ignorance. Can anyone explain this behavior or at least
> point me to a place to educate myself?
>
> Many thanks, the script I'm running is below.
>
> --------
> #!/usr/bin/env ruby
> require 'rubygems'
> require 'hpricot'
>
> def other_task
>   a = []
>   9999.times  do
>     a << "12345678901234567890123456789012345678901234567890" * 100
>   end
>   nil
> end
>
> def task
>   # 500K xml data
>   data = File.readlines("very_large_output.xml").to_s
>   temp = Hpricot.XML data
>   nil
> end
>
> puts "In task"
> 10.times {|i| task; p i}
> puts "In other task"
> 100.times {|i| other_task; p i}
> puts "In task (Should memory go down?)"
> 100.times {|i| task; p i}