Martin Bosslet <Martin.Bosslet / googlemail.com> wrote:
> Eric Wong wrote:
> > Martin Bosslet <Martin.Bosslet / googlemail.com> wrote:
> >  > This is related to the proposal in [ruby-core:41321][1].
> >  > 
> >  > I'd like to take advantage of streaming IO in an extension I am
> >  > working on. The problem I'm having is that I don't want to call
> >  > IO#read on the rb_funcall level because that would kill the
> >  > performance due to wrapping the bytes into Ruby objects back and
> >  > forth again.
> >  
> >  Is starting with Ruby String objects (with binary encoding) and then
> >  having read(2)/write(2) hit RSTRING_PTR not possible?
> 
> You mean reading String chunks from the underlying IO? I'm afraid not.
> The only way I could right now is calling the Ruby methods for 
> IO#read/write using rb_funcall. But there's a lot of overhead involved, 
> VM roundtrip plus lots of short-lived objects that trigger GC. It would 
> likely end up being slower than the current ASN1.decode, a situation I'd 
> like to avoid.

You can avoid short-lived objects by passing Strings as the second
argument to IO#read-like methods:
`
	buf = ""
	while r.read(16384, buf)
	  w.write(buf)
	end

Without GC calls happening, I don't expect significant overhead from the VM.

> >  If you're working with sockets/pipes, I would reckon not (Ruby already
> >  defaults to IO#sync=false on sockets/pipes when writing).

Err, typo on my part, IO#sync=true is the default (meaning no userspace
buffering is the default).

> Still, I am wondering if there is the need for a low-level C API for doing
> IO on Ruby IO objects, or is the "clean cut approach" using the file descriptor
> directly the recommended solution in any case?

I recommend directly working off the file descriptor.  The only tricky
part is making sure there's nothing in the userspace buffers beforehand.

You'd probably need to call IO#rewind/IO#seek to sync read buffers up
and IO#flush (and then IO#sync=true) for write bufffers.