Issue #13085 has been updated by Eric Wong. File 0001-basicsocket-Linux-workaround-for-excess-garbage-on-w.patch added At least on Linux, this is probably the best performance we can get with sockets. Unfortunately, this does not affect writes to pipes (write_nonblock is OK for me, at least), and regular files (where we really need to release GVL) --------8<------- Subject: [PATCH] basicsocket: Linux workaround for excess garbage on write Linux allows us to use the MSG_DONTWAIT flag on sockets to do non-blocking send(2) without side effects of modifying the underlying file flags. This allows us to replace the generic IO#write and avoid garbage creation on GVL release: <https://bugs.ruby-lang.org/issues/13085> We now only release the GVL when IO cannot proceed. While IO#write_nonblock may be used for pipes and sockets portably, that has the side effect of changing the file flag via fcntl. MSG_DONTWAIT under Linux has no side effects on the socket. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. I HATE IMPLICITLY SHARED MEMORY. ---------------------------------------- Bug #13085: io.c io_fwrite creates garbage https://bugs.ruby-lang.org/issues/13085#change-62512 * Author: Eric Wong * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: * Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- Relying on rb_str_new_frozen for unconverted strings does not save memory because copy-on-write is always triggered in read-write I/O loops were subsequent IO#read calls will clobber the given write buffer. ```ruby buf = ''.b while input.read(16384, buf) output.write(buf) end ``` This generates a lot of garbage starting with Ruby 2.2 (r44471). For my use case, even `IO.copy_stream` generates garbage, since I wrap "write" to do Digest calculation in a single pass. I tried using rb_str_replace and reusing the string as a hidden `(klass == 0)` thread-local, but `rb_str_replace` attempts CoW optimization by creating new frozen objects, too: https://80x24.org/spew/20161229004417.12304-1-e / 80x24.org/raw So, I'm not sure what to do, temporal locking seems wrong for writing strings (I guess it's for reading?). I get `test_threaded_flush` failures with the following: https://80x24.org/spew/20161229005701.9712-1-e / 80x24.org/raw `IO#syswrite` has the same problem with garbage. I can use `IO#write_nonblock` on fast filesystems while holding GVL, I guess... ---Files-------------------------------- 0001-io.c-io_fwrite-temporarily-freeze-string-when-writin.patch (2.6 KB) 0001-v2-io.c-io_fwrite-copy-to-hidden-buffer-when-writing.patch (2.9 KB) 0001-basicsocket-Linux-workaround-for-excess-garbage-on-w.patch (5.54 KB) -- https://bugs.ruby-lang.org/ Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe> <http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>