------extPart_000_00F3_01C4973C.2D830650
Content-Type: text/plain;
	charsetso-8859-1"
Content-Transfer-Encoding: 7bit


"Charles Hixson" <charleshixsn / earthlink.net> schrieb im Newsbeitrag
news:41417F73.5060002 / earthlink.net...

> Marshall is the wrong answer.  The hash will be limited to around 1500
> items by flushing.  The database of which it is a partial updated mirror
> will likely grow to around 5,000,000 items.  The flushing process
> detects all dirty items in the hash and passes them to another routine
> which either updates an existing item or adds a new one.

So if I understand you correctly it's like this: you have a hash data
structure in mem that keeps some data among that data that is not yet
present in the DB.  You want to store all dirty data in a file to make
sure that in case of a crash you don't loose anything.  From time to time
you write the dirty stuff into the database and if that succeeds you clear
the temp disk storage.  I'll attach something that shows how I image this
could work.

> Actually, there will likely be several (?) hash tables simultaneously in
> the eventual implementation, and each one will need to implement a
> different version of this procedure.  Fortunately, the records
> structures are both orderly and consistent, so I won't need the kind of
> flexibility that marshall implies.  I may even eventually translate this
> into a compileable language after I get everything working, for the
> increase in speed that's available.  (My plan is 1) first get it
> working, 2) second, speed it up.)

That's the way how it should be ("premature optimization..."). :-)

>  If this is adopted I'll probably use
> D (DMD), as that seems the best of the current compileable languages.
> (I wonder if Ruby-inline could handle D code?  Well, no rush.  That's a
> long ways off yet.)

Just a wild idea:

functor  ::compile <<EOF
  D code here
EOF

functor.call( "foo", "bar" )

Of course you'd have to compile the code into a shared lib and dynamically
load it.  But it sounds feasible IMHO.  Maybe it's a good idea to provide
a framework for this, so integration of other languages becomes easier.

Kind regards

    robert

------extPart_000_00F3_01C4973C.2D830650
Content-Type: application/octet-stream;
	nameersistent_hash.rb"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filenameersistent_hash.rb"



class DirtyHashSupport < Hash
LIMIT = 10

def initialize(file)
super
@file = File.open(file, "ab+")
@persist = true
no_persist { update( store_persistent_to_db ) }
end

def []=(key, val)
super

# print "added key=", key.inspect, " val=", val.inspect, "\n"

if @persist
print "dumping key=", key.inspect, " val=", val.inspect, "\n"

Marshal.dump([key, val], @file)
@file.flush
@dirty += 1

store_persistent_to_db if @dirty > LIMIT
end
end

def close
if @file
@file.close
@file = nil
end
self
end

private

def store_in_db( key, val )
print "Storing key=", key.inspect, " and val=", val.inspect, "\n"
end

def store_persistent_to_db
@file.seek 0
stored = {}

begin

loop do
key, val = Marshal.load( @file )
store_in_db( key, val )
stored[key]=val
end
rescue EOFError
# done
end

@dirty = 0
@file.truncate 0
@file.seek 0

return stored
end

def no_persist
@persist = false
begin
yield
ensure
@persist = true
end
end
end


dh = DirtyHashSupport.new( "/c/temp/dh.bin" )
p dh

dh["foo"]="bar"
dh["fox"]="bary"
10.times {|i| dh[i]=i+100 }
p dh

dh.close

dh = DirtyHashSupport.new( "/c/temp/dh.bin" )
p dh
-----extPart_000_00F3_01C4973C.2D830650--