On Mon, 19 Dec 2005, Charles Ballowe wrote:

> On 12/18/05, gwtmp01 / mac.com <gwtmp01 / mac.com> wrote:
>>
>> I hope you don't mean reading/writing to the raw device.   I would
>> do a whole lot of other things before I would drop down to
>> raw device access. Premature optimization is not a good thing.
>>
> nah... it would require filesystem interfaces to the block mapping in
> the inode - I don't know if such things exist and they certainly
> aren't portable, but it seems like it could be a very efficient way to
> drop data out of the middle of the file. Probably over-optimizing
> though.
>
> Of course if the records were fixed sized and block aligned, the
> shuffling would be pretty efficient and the extra level of
> optimization would likely be overkill.
>
> -Charlie

if your records are fixed size you'd be mad to takle this application without
considering bdb (berkeley db) and using it's record database file format.  this
interface would make modifying the data extremely quick.  also, if the records
are in fact fixed size, using mmap is the cheapest way:

   [ahoward@jib ahoward]$ cat a.rb
   require "yaml"
   require "mmap"

   records =
     %w( a b c ),
     %w( 0 1 2 ),
     %w( x y z )
   open("records", "w"){|f| f.write records.join}

   y "records" => IO::read("records")

   mmap = Mmap::new "records", "rw", Mmap::MAP_SHARED

   record_0 = mmap[0,3]
   record_1 = mmap[3,3]
   record_2 =  mmap[6,3]
   mmap[3,3] = record_2  # move record down
   mmap[6 .. -1] = ""  # truncate

   mmap.msync
   mmap.munmap


   y "records" => IO::read("records")


   [ahoward@jib ahoward]$ ruby a.rb
   ---
   records: abc012xyz
   ---
   records: abcxyz


it's tough to do io better than the kernel...

regards.

-a
-- 
===============================================================================
| ara [dot] t [dot] howard [at] noaa [dot] gov
| all happiness comes from the desire for others to be happy.  all misery
| comes from the desire for oneself to be happy.
| -- bodhicaryavatara
===============================================================================