Randy Kramer <rhkramer / gmail.com> writes:

> I'm really not trying to start a flamewar.  Some of the statements made in 
> this thread seem completely contrary to what I thought I knew, and 
> counter-intuitive as well.

Based on what you wrote below, the statements I made are in
agreement with what you thought you knew for the most part.

> On Sunday 16 October 2005 03:02 am, Yohanes Santoso wrote:
>> Memory leak is tricky to show since it requires one to prove that
>> there is an allocation that is not free-ed by the time the process
>> ends.
>
> I didn't/don't think so.  Unused memory allocations that are not freed while 
> the process is running seem to be the very definition of a memory
> leak:

I can go with this definition. The difference between this and mine is
the word 'unused'. In a language with GC, 'unused' is an expensive
(difficult too) proposition to automatically detect. If the GC is
aggressive, the number of unused allocations has a higher probability
of being smaller than a conservative GC, and Ruby has a conservative
GC. There is this trade-off of being agressive & more expensive
vs. conservative & cheaper.

> Result of Googling [define:"memory leak"]:

I agree with all these statements, and they do not contradict my
statement either.

In addition, I'd like to emphasis this:

> Memory leaks are often thought of as failures to release unused memory by a 
> computer program. Strictly speaking, it is just unneccesary memory 
> consumption. 

unused allocation is just an alias for unnecessary memory consumption.

> A memory leak occurs when the program loses the ability to free 
> the memory. 
> en.wikipedia.org/wiki/Memory_leak

No evidence yet that the GC loses the ability to free the
memory. Proofing this amounts to proofing that an allocation is not
freed by the time the process ends. Which is exactly my statement.

> I'd call it a memory leak even if the allocation is freed when the process 
> ends.  Consider processes that (should) run continuously.

Then this really falls under the category 'unnecessary memory
consumption' as per the reference you gave.


> That may be true (I really don't know)--but:
>    * if increasing VmSize corresponds to a slowdown in my machine, I'd call it 
> a pretty good "indicator"

Try tracing through leak not_immed. You will see this peculiar
variable called 'hole'. hole holds a pointer to the last allocation
performed. It's there to show that allocated memory is not necessarily
returned to OS.

By the time it gets to the 'unwinding' part, there are exactly 8194
allocations (8193 for array&elements, 1 for hole). By the time the
execution returns back to main and displays the vminfo, there are
exactly 8193 free(). There is still 1 allocation that is not
freed. Yet, the VmSize shows that the other 8193 free() does not
result in the memory being returned to the OS.

What I was saying to Eric Mahurin was, the ruby gc, even being
conservative as it is, could have free()ed all the unnecessary
allocations yet, there could still be one allocation that would result
in every freed allocations in not being returned to the OS. In this
case, a program really cannot do anything. 

So a large VmSize value could have meant anything: it could have meant
that the gc is too conservative, it could have meant the gc is buggy,
it could have meant the gc has free()ed all allocations saves one, it
could have meant ..., etc.

>    * what can you use as a better indicator

object count would be a better indicator.

def count_objects
    object_count=0
    ObjectSpace.each_object{ object_count += 1}
    object_count
end

Had Eric used this, I would not have been involved at all in this
thread.

>> In some OS, VmSize is an always increasing 
>> number. 
> Out of curiosity (and the intent to try to avoid such), which OS?

My first RL experience with this is in 1997 while working with a
HP-UX. Memory allocated was never returned to the OS. If you were to
modify leak so that it free() hole immediately and run it on that OS,
the VmSize would not decrease, although the VmRes would (see below).

> better clarify your statement--are you saying that in some OS, VmSize does 
> not decrease even if a program is stopped?

VmSize depends on the existence of the process, as such, if the
process dies, VmSize no longer exists too. But I'm guessing that your
real question is whether the OS will reclaim the memory when a process
dies. In all *NIX OS, this is a true statement. Some versions of win95
(win3.11 too?) do not. Make a program that allocates some memory, kill
the program, run it again, kill it again, until you see allocation
failure.

>> Memory allocated in a process is not returned to the OS until 
>> the process dies.
>
> Again, seems like the very defintion of a memory leak.

'unnecessary memory consumption' :)
 
> It is true that if the memory allocated to a process is returned to the OS 
> when the process dies, that gives you a workaround for the memory leak, but 
> it is still a memory leak.

Yeah, this depends on your definition of memory leak. It is an example
where terms have taken new meanings (incorporating the concept of
'unnecessary memory consumption'). Another example would be the word
'scalable' which today would be taken as 'high-performant' (thanks
zedas) by many people, or a more extreme example would be 'hacker'
which has became to mean the opposite, 'the bad guy'.

> occasionally close kmail and epihany and get back a lot of memory (and make 
> my system run a lot better.)

In *NIX OS, what you should really be watching for is not VmSize,
rather VmRes[ident] which shows you how much RAM your program is
really taking. The diff of VmSize-VmRes would be in swap where it
could just stay there until the process ends.

This certainly is true in long-running processes. For example, on my
desktop system which has been running for 3 months (and some
applications have been running that long too: emacs, kde, konq), the
total VmSize of all processes is around 3GB, way more than the amount
of RAM I have (around 700MB), yet those apps still respond crisply,
and the total VmRes of all process is still under 400M.

> regards,
> Randy Kramer

Regards,
YS.