On 20/10/2007, Jay Levitt <jay+news / jay.fm> wrote:
> On Sat, 20 Oct 2007 04:55:18 +0900, Michal Suchanek wrote:
> > On 19/10/2007, M. Edward (Ed) Borasky <znmeb / cesmail.net> wrote:
> >> Now the killer is this: the platform (hardware and OS) designers make a
> >> bunch of compromises so that you can get "acceptable" performance for a
> >> lot of different languages -- compiled or interpreted, static memory
> >> allocation or dynamic memory allocation, explicit memory
> >> allocation/deallocation or garbage collection, etc. And the language
> >> designers make a bunch of compromises so that you can get "acceptable"
> >> performance on modern operating systems. It's almost as if the two types
> >> of designers communicate with each other only every fifteen years or so.
> >
> > I cannot imagine what else you can do when you want an OS that runs
> > pretty much all languages. All that the OS can do is hand out pages,
> > and only the language runtime can manage the data inside those pages.
> > Unless you tailor the OS to one specific language or virtual machine
> > you cannot get anything more.
>
> There was a paper a few years ago on the idea of a GC that could
> communicate with the virtual memory manager, so that it could sweep pages
> when they happened to be already swapped in, rather than intentionally
> pulling them in purely to do the sweep.  Or something along those lines;
> the key is that right now, you have no way to know if accessing an address
> will produce a page fault.

You could probably see your memory map and your pagefaults, after all
it's your memory. I am not sure it would buy you anything, though. You
have to visit all pages to perform the garbage collection. You could
possibly optimize the order a bit but the pages will be probably
interlinked in weird ways and unless you do something very clever you
can easily get lost.

>
> It makes a lot of sense.  I always assume that big VSZs (at least in
> garbage-collected apps) are bad because they lead to eventual paging, but I
> don't know nearly enough to prove or even investigate that.
>

It's inevitable if the garbage collector accesses all the memory, and
most do. You could have reference counting that only discards stuff on
currently mapped pages (which is what reference counting does most of
the time anyway - it only decreases the count for stuff that is
accessed). However, reference counting is not sufficient to get rid of
garbage so you have to sift through the whole heap eventually, and
that pages it in in its entirety.

Some applications might get away with large VSZ and no paging but it
proves that the memory is in fact useless and probably was not
discarded only because of memory management errors - leaks.

Thanks

Michal