On 5/17/07, Joel VanderWerf <vjoel / path.berkeley.edu> wrote:
> Brian Candler wrote:
> ...
> > Or you could just keep keys in RAM and use separate disk files for each
> > object.
>
> That's sounding sorta like fsdb[1]. I'm not sure I can recommend it in
> this case, though, since you pay for Marshal.dump every time you write
> an object into the database, plus locking, plus IO. I'd expect
> FSDB::Database::[]= will be orders of magnitude slower than Hash#[]= .
>
> It will be interesting to hear any solution using mmap....
>

You could do a mmap solution. Modify Hash such that []= does a
Marshal.dump of your object, stores the object into the mmap, and then
that memory location is stored in the Hash instead of the object.

[] must also be modified to take the memory location, Marshal.load the
object from the mmap, and then return the object.

The hard part of this is doing the memory management of the mmap --
when an object is deleted from the hash, removing it from the mmap;
consolidating unused mmap regions; etc.  All the standard MMU stuff
you normally don't have to deal with in Ruby.

It would be much easier to implement if all the objects being stored
in the Hash were guaranteed to be the same size. Then you would just
need an free/allocated array to keep track of what can go where in the
mmap.

Blessings,
TwP