On Sun, Dec 5, 2010 at 7:15 PM, Diego Virasoro <diego.virasoro / gmail.com> wrote:
>
> The data can be thought of as hierarchical tables: i.e. tables in the
> SQL sense with *some* entries corresponding to entire new tables, and
> the others either strings or numbers (integers and float). I have only
> access to one computer, though with multiple processors, so I don't
> have any need for distributing the data. Finally I'd like speed, but
> given the database size I can't keep all the info in memory.
>
> The kind of operations I need to do involve getting slices of tables
> (all the rows but a subset of columns), often in all the "subtables"
> I've got. I also need to add a few rows to each table when the
> simulation is running (but not for analysis) however the rate is
> rather low so I don't expect this to be much of a constraint.
>
> Could people with more DB experience give a few suggestions? I've read
> of MongoDB, CouchDB, MySQL and PostgreSQL... and to be honest I am
> pretty lost, so I am sending this in the hope some fellow Rubyist can
> help a poor DB newbie. ;)

SQLite[0] might be worth a look, especially if you can split your data
across several instances of SQLite.

The key difference of SQLite to things like MySQL, PostgreSQL, and
others is that SQLite is not a server, in that it doesn't offer remote
access, which you say you don't need, so that is not an issue.

Additionally, you have no overhead of administering (or at least
setting up) a database server and its users, and SQLite is lighter on
your resources. ;)

If you have data that doesn't lend itself well to storing them in the
rather rigid confines of SQL, take a look at the NoSQL DBs like
CouchDB or MongoDB. All of those (and the SQL engines) have good to
great Ruby bindings, so you can use them with relative ease.

[0] http://sqlite.org/

-- 
Phillip Gawlowski

Though the folk I have met,
(Ah, how soon!) they forget
When I've moved on to some other place,
There may be one or two,
When I've played and passed through,
Who'll remember my song or my face.