Lothar Scholz wrote:
> Hello Daniel,
>
>>>
>>> The analysis builds a profile in memory of over 3.6 GB of
>>> data on disk.  On the Solaris box, it takes about 35 mins and
>>> uses about 700 MB of RAM.  It would not complete on the
>>> windows box using the full data set, bombing with "failed to
>>> allocate memory (NoMemoryError)".  There was nearly 800 MB of
>>> RAM free on the windows box as well as having a 4 Gig swap
>>> available.
>>>
>>> Is windows that inefficient with memory allocation or is this
>>> a ruby implementation issue on windows?
>
>> I suspect mostly the latter, with a little bit of the former mixed in
>> for good measure.  I'm pretty sure the GC algorithm is different on
>> Windows, though I can't remember the details now.
>
> No its exactly the same. Remember that the ruby GC has an extremely
> dumb implementation.
>
> The problem is MSVC runtime. It is well known to be fragmenting. There
> are many drop in replacements. Programs with large datasets should
> never use the default malloc/free.

Also I believe the limit for user process memory is lower on 32 bit
windows boxes than on 32 bit Solaris systems due to the way windows
relocates DLL's etc.

Kind regards

    robert