On Thu, 11 Oct 2007 22:15:04 +0900, Blackie <mudsweatandears / gmail.com> wrote:
> During "other_task" I can see the usage rise *and fall* so I know the
> OS is reclaiming *some* memory in places...but the part I don't
> understand is why it isn't returning to "tasks" original base of 50M
> when only "task" is running.

This is not actually a problem specific to Ruby, but applies to the
majority of software which does not use a compacting allocator (this
includes C, with malloc/free[1]).

Typically, memory is allocated in blocks from the OS, and parceled out
to individual objects from there.  When there are no more active objects
in a block, the block could theoretically be returned to the OS.  In
practice that can be hard to do, since individual "stragglers" can
prevent entire blocks from being reclaimed.  The block can still be
reused within the program to allocate new objects, but it may not be
possible to return to the OS while the program is still running[2].

Ruby does make a reasonable effort to return unused blocks ("heaps" in
the parlance of gc.c) to the OS[3], when it is possible to do so.  But
it is not always possible to do so.

> I hate to sound dense, but I need to convince some fairly hard-headed
> sysadmins.

Do they use Perl?  Perl 5 does not even try to return memory to the OS.

-mental

[1] malloc/free can be even worse, if implemented using brk/sbrk,
    since a single live object at a high address can "pin" the entire
    rest of the heap below it, not just a single block

[2] a process' memory is always reclaimed by the OS once it exits

[3] see free_unused_heaps in Ruby's gc.c