On Sat, 26 Aug 2006, Zed Shaw wrote:

> Ara, this is uncool.  Seriously man, don't e-mail me personally and then
> e-mail the list the exact same e-mail.  Pick one and fight your battle
> there.

hi zed - calm down a minute!  ;-)  i have do idea what my mailer did but
suspect it has something to do with all the recipients and the fact that it
has to swtich roles to post to rails/ruby-talk.  in any case, it wasn't by
design.

> As I mentioned to you before, the evidence shows you are wrong.

but my evidence shows you are!  i'm merely pointing out it's unclear at this
point.

> Sure you've cooked up a script that has a memory leak with Sync, but that
> script doesn't operate the way Mongrel does.  The sample I developed does
> operate the way Mongrel does.  It's called a "bug reduction".

i think it's 'bug production' though - for instance, your loop that starts
'1000' threads starts a random number of them, your gc call, as bob showed,
doesn't seem to be quite enough (not that it's your fault).  and there are
race conditions/edge cases with the os memory subsystem.

> I'm not going to test the leak in Mongrel with a reduction that doesn't
> simulate Mongrel.

but you could change the code on mongrel right!?  bear in mind you have not
just been talking about changing your code, but advocating changing ruby and
all ruby software - you have to back up that sort of thing!

> Also, as I said before, this shows the leak:
>
> http://pastie.caboo.se/10194

but it doesn't show a leak on my system, on bob's system, or on ed's system?
just tons of memory usage.  are you ignoring the fact that no one else who has
run it shows a leak either?  how about the fact that electric fence, mpatrol,
and mtrace also show no leak?  how about the fact that very few people have
had issues with mutex and memory?

> And this script, with just 3 lines changed to use Sync shows it's fixed:
>
> http://pastie.caboo.se/10317
>
> With graphs even Ara!  Graphs!  We ran these tests for 30-40 minutes with
> hundreds of thousands of threads being cycled during the tests.

well, you can graph my numbers too and they'll show a sawtooth for mutex and
linearly increasing line for sync, at least for the number of iterations i
posted.  which is, of course, exactly the opposite of what you showed.  btw,
i'd be glad if both of us could be proven wrong.

> Not to mention about 8 other people switching to Sync report their leaks
> fixed, our own test script showing it's fixed, Mongrel tests showing it's
> fixed, several other frameworks showing it, and you can't argue with the
> evidence.

i'm sure that's not true.  what has happened is that they've shown a reduction
in memory usage.  i don't think anything else has even been able to confirm
your code leaked - just that i grew large - eg had a bad memory
allocation/free cycle.  reads bob's post for more info.

> If your script has a leak then fine, just don't do that.

exactly my point too!

> Ultimately though the ruby lang guys need to fix this because either way,
> there's leaks.

here i disagree.  i'm not sure either of us has show a leak in __ruby__.  we
may have demonstrated a bad memory usage pattern for certain memory subsystems
- this is far fetched, read comp.unix.programmer and you can see thousands of
threads on this subject.  it's a very frustrating one for many people.

> For now, Mongrel is not leaking and I'm happy with that.

granted.  however - screaming that everyone should re-write their code which
has used anything other than sync is just a bit extreme considering no one
else can show a leak - only badly behaving scripts - isn't it!?

> Now, I'd appreciate it if you'd maybe spend your energy looking into the
> ruby source to find this leak rather than bothering me about it.

the whole reason i'm posting is that i too have had issues with mutex/sync etc
and not solved it to my satisfaction.  if you are satisfied fixing a heisenbug
and calling it good then fine.  however the other reason i was posting is that
i've fixed hundreds of bugs in long running near realtime systems where the
coder wrote somthing they didn't really understand that seemed to 'fix' the
problem, only to create new ones and release a program that's even more
difficult to debug.  it's very suprising how many peices of code work out of
luck but really shouldn't - if you're ok with that then fine.  i just don't
happen to be.

regards.

-a
-- 
to foster inner awareness, introspection, and reasoning is more efficient than
meditation and prayer.
- h.h. the 14th dalai lama