On Mon, 23 Oct 2006, snacktime wrote: > I've tested out a couple of ways of storing a queue structure and > wondered if others had some better ideas. I tried a berkeleydb based > queue using it's native queue structure, which gives the best > performance so far but it's for unix only. I also have a file based > queue where every message is just stored in a sequentially numbered > file. Almost as fast as berkeleydb and works anywhere. Just for > giggles I tried sqlite, and then immediately deleted it. 6-8 tps with > sqlite compared to around 1000 with berkeleydb/flat files. > > Another alternative I was thinking of is a single file storage of some > type, maybe fixed length or even variable length records. Something > that's portable. But I'm thinking that could get complicated fairly > quickly. > > Any other ideas? why not: harp:~ > cat a.rb require 'dbm' n = 10000 db = DBM.open 'db' at_exit{ db.close } kvs = Array.new(n).map{ [ k = rand(n).to_s, v = rand(n).to_s ] } a = Time.now.to_f; kvs.each{|k,v| db[k] = v}; b = Time.now.to_f elapsed = b - a puts("elapsed : #{ elapsed }") puts("tps : #{ n / elapsed }") harp:~ > ruby a.rb elapsed : 0.220124006271362 tps : 45428.9387576941 that's damn fast: it runs at nearly the exact same speed on my windows box too. btw, dbm will in fact be backed by bdb on any newish *nix dist, and ndbm for the windows one-click installer - either way though, you get it in the standard lib and, even if you decide to marshal objects it's still extremely fast, easy to use, and disk based. it should be very robust, esp in the case of a bdb backed impl since bdb is acid. regards. -a -- my religion is very simple. my religion is kindness. -- the dalai lama