On Friday 07 December 2001 01:35 pm, James Britt (rubydev) wrote:

> What I'm trying write is a Source class for REXML's stream parser.
> It should behave as if it were pulling the XML from a file.  REXML comes
> with a File-based Source class that reads blocks of characters from the
> file as the stream parser progresses.  The file can be arbitrarily large;
> the parser shouldn't know or care, assuming the parser isn't holding on to
> the data once it's been processed. One of the touted benefits of SAX-like
> XML processing is that it allows you to process large XML sources without
> having to hold all the XML in memory.
>
> In my DbSource, when the stream parser asks for more text, the DbSource
> class goes and gets more data (e.g., fetch_many(50)), converts it to XML,
> and hands back a string.  So, from the stream parser's point of view, the
> XML is just another file.

You may want to consider the BerkeleyDB (ruby/bdb ?), which gives a high 
performance Hash (among other things). Just a bunch of name/value pairs, 
where the name and/or the value can each be up to 4Gb long.

Good disk management, fast, and uses shared memory by default, so works well 
as a persistent intercommunications scheme between programs.

And even better: no database server (unless you want one, like if you have a 
remote database)!

Locking, secondary keys, etc. if you need them.

Unless you need SQL (it doesn't sound like you do), that is.

(note: MySQL is built on top of BerkeleyDB).

-- 
Ned Konz
currently: Stanwood, WA
email:     ned / bike-nomad.com
homepage:  http://bike-nomad.com