[snip somepne else's comment, similar to mine,
about varying probabilities]

> Yes, this is what I said in my second message.

Oops, I failed to see that comment. I see it now.  :)

This makes me think of another possibility besides
a real index file...

I'm assuming (perhaps incorrectly) that a seek and a
read of a specified number of bytes is faster than a
read of a single line of unknown length.

A rather theoretical question, since the technique I'm 
proposing might waste almost as much as it saves...

Suppose on the first line (line 0) you store the length 
of line 1.

Then on each line you store the length of the NEXT line
in the same way.

You can hop through the file quickly in this way, no?
On each intermediate line, you need only a read a single
number (perhaps three digits fixed-length).

Just a random neuron firing...

Hal Fulton