On Fri, 2006-08-25 at 14:33 +0900, ara.t.howard / noaa.gov wrote:
> On Fri, 25 Aug 2006, Zed Shaw wrote:
> 
> > Hi Folks,
> >
> > Sorry to get your attention.  :-)
> >
> > There's a very strange problem with Mongrel where if Threads are created
> > because of the Mutex around Rails dispatching, then lots of ram gets
> > created that never seems to go away.
> >
> > I boiled the problem down to this:
> >
> > http://pastie.caboo.se/10194
> >
> hi zed-
> 
> i don't think you have a leak.  try running under electric fence (ef).  when i
> do i clearly see the memory rise from 1->20% on my desktop, and then decline
> back down to 1%, over and over with no reported leaks.  the cycle matches
> the logging of the script perfectly.
> 
> here's the thing though, when i don't run it under electric fence i see the
> memory climb to about 20% and then stay there forever.  but this too does not
> indicate a leak.  it just shows how calling 'free' in a process doesn't really
> release memory to the os, only to the process itself.  the reason you see the
> memory vary nicely under ef is that it replaces the standard malloc/free with
> it's own voodoo - details of which i do not understand or care too.  the
> point, however, is that it's 'free' which is doing the 'leaking' - just at the
> os level, not the process (ruby) level.  we have tons of really long running
> processes that exhibit the exact same behaviour - basically the memory image
> will climb to maximum and stay there.  oddly, however, when you tally them all
> up the usage exceeds the system capacity plus swap by miles.
> 

Nope, I can't agree with this because the ram goes up, the OS will kill
it eventually, and if I remove the guard the ram doesn't do this.

And where are you getting your information that free doesn't free
memory?  I'd like to read that since all my years of C coding says that
is dead wrong.  Care to tell me how malloc/free would report 80M with
Mutex but properly show the ram go down when there is no-Mutex?

And why Linux would kill processes if the ram get too high?  Why whole
VPS servers crash?  I mean if this ram was just "fake" reporting (which
is very hard to believe) then why are all these things happening?

So please, point me at where in the specifications for malloc/free on
Linux it says that the memory reported will be high even though free and
malloc is called on 80M of ram slowly cycled out, and that linux will
still kill your process even though this ram is not really owned by the
process.


-- 
Zed A. Shaw
http://www.zedshaw.com/
http://mongrel.rubyforge.org/
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.