On Wed, 2004-10-27 at 02:00, ts wrote:
> >>>>> "M" == Markus  <markus / reality.com> writes:
> 
> M>      I'm playing with a patch which should handle up to 4 billion lines
> M> with no memory penalty at all (or perhaps a very slight gain),
> 
>  You can perhaps handle 4 billion lines, this will not change the fact that
>  the number of nodes in a block is limited.
> 

     *laugh*  No, and I am not intending to tackle that soon.  (Though I
suppose it could be possible to have a billion line program with very
few nodes--if, for example, there were large here-doc strings being
managed by a small amount of code).  But the real reason for the 4
billion is of course that I'm using a long so as to maintain alignment
within the struct. 

     Technically I should say "I'm trying to..." since it isn't quite
working.  The basic idea is this:

      * RNODEs have a header (constant length part) and one of three
        variant parts, called u2, u2. and u3.

      * I came up with a patch (see earlier in this thread) that neatly
        handled line numbers greater than 8k; I added a long to the
        header.  (Previously, they had been packed into bits in the
        flags field).  This maintained double-word-alignment, but grew
        the structure by four bytes.

      * u3 is the longest variant part.  If it were four bytes shorter,
        the whole structure would be the same length as it had been
        before I my patch.

      * u3 contains a long status field that is only ever checked for
        0,1,2, or none of the above.  If it were moved into the flag
        bits previously used by the line numbers (it would only take two
        of the thirteen) balance and harmony would be restored to the
        chaos.

But the devil is in the details.  It seems (I have not gotten to the
bottom of it) that sometimes fields in the various variants are
referenced to modify or access the fields in the others (making use of
the overlap).  Not really knowing C, I'm having a hard time tracking
them down--using grep, detective work, and a blunt stick where I suspect
others would use gdb and a surgical knife, or some other tool I'm not
even aware of.

     It is of course also possible that I have misdiagnosed the problem.

         -- Markus