Martin Bosslet <Martin.Bosslet / googlemail.com> wrote:
> In the general case, read_all grows its buffer linearly by just the
> amount that is currently read from the underlying source. This results
> in a linear number of reallocs, It might turn out beneficial if the
> buffer were grown exponentially by multiplying with a constant factor
> (e.g. 1.5 or 2), thus resulting in only a logarithmic numver of
> reallocs.

I think growing the buffer exponentially makes sense.

I would enforce a hard limit (probably <= 8 MB) for each growth,
to:

1) discourage read_all() for large files, it's very wasteful and
   usually hurts performance

2) prevent memory exhaustion for edge cases (especially on 32-bit)