Issue #13597 has been updated by akr (Akira Tanaka).


I think that it's possible to call remalloc when
"outbuf" argument is not supplied to read_nonblock.

It makes possible to
automatically reduce memory with remalloc (without supplying "outbuf") and
reuse buffer without remalloc (with supplying "outbuf").


----------------------------------------
Misc #13597: Does read_nonblock call remalloc for the buffer if does it just set the size attribute
https://bugs.ruby-lang.org/issues/13597#change-66373

* Author: emilys (Emily Stolfo)
* Status: Open
* Priority: Normal
* Assignee: 
----------------------------------------
Hello

I've observed that a lot of memory gets allocated and wasted when read_nonblock is called for a number of bytes much larger than is actually read from the socket.
This line https://github.com/ruby/ruby/blob/0130bb01baed404c0e3c75bd5db472415a6da1d3/io.c#L2686 appears to eventually only change the heap size value here https://github.com/ruby/ruby/blob/144e06700705a3f067582682567bc77b429c4fca/string.c#L104 but does not call remalloc.

I see this request to allow an offset to be passed to read_nonblock:
https://bugs.ruby-lang.org/issues/11484

but until that is implemented, how do you recommend efficiently asking to read a large number of bytes from a socket? If I'm not mistaken, if I request 16000000, but only read 1000000, the buffer that has been allocated in io_read_nonblock for 16000000 doesn't seem to be resized.

Would you recommend instead requesting a more predictable number of bytes, closer to the default system value (SO_RCVBUF, for example) in each call to read_nonblock?

For context, this pull request against the MongoDB Ruby driver has lead me to this investigation. https://github.com/mongodb/mongo-ruby-driver/pull/864

Thank you in advance
Emily





-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>