Issue #14372 has been updated by jrafanie (Joe Rafaniello).


I did a small change to see how the number of Pathnames in the `$LOAD_PATH` changes the leak amount at the script's completion.

It looks like the memory leak is linear:
1  74.6 MB
2 149.5 MB
3 214 MB
4 290 MB
5 353.6 MB
9 575.4 MB
10 650.6 MB

Here's the script:

~~~ruby
require 'pathname'

puts Process.pid

puts ARGV[0]
(ARGV[0] || 1).to_i.times { $LOAD_PATH.unshift(Pathname.new(__dir__) ) }

dot      = "."
filename = "ostruct"
1000.times { 1000.times { require filename }; print dot; GC.start; }

STDOUT.puts "exit?"
STDIN.gets
~~~

Additionally, I measured the time to do the requires only changing the number of Pathnames in the `$LOAD_PATH`:

1 Pathname, 6.1s
2 Pathname,  8s
3 Pathname, 10.1s
4 Pathname, 12.2s
5 Pathname, 13.4s
9  Pathname, 20.3s
10 Pathname, 22.2s

~~~ruby
require 'pathname'

puts Process.pid

puts ARGV[0]
(ARGV[0] || 1).to_i.times { $LOAD_PATH.unshift(Pathname.new(__dir__) ) }

dot      = "."
filename = "ostruct"
1000.times { 1000.times { require filename }; print dot; GC.start; }
~~~

----------------------------------------
Bug #14372: Memory leak in require with Pathnames in the $LOAD_PATH in 2.3/2.4
https://bugs.ruby-lang.org/issues/14372#change-69631

* Author: jrafanie (Joe Rafaniello)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-darwin16]
* Backport: 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
There is a memory leak that we have found on ruby 2.3.6 and 2.4.3 that happens on Mac OSX and Linux.  Ruby 2.2.6 and 2.5.0 do not leak.  We have not tested other platforms.

If `$LOAD_PATH` contains one or more Pathname objects, `require` without a fully qualified path, such as `require 'ostruct'`, will leak if you do this require many times.

For example, the following script will leak very quickly on ruby 2.3.6 and 2.4.3:

~~~ruby
require 'pathname'

puts Process.pid
$LOAD_PATH.unshift(Pathname.new(__dir__))

dot      = "."
filename = "ostruct"
1000.times { 1000.times { require filename }; print dot; GC.start; }
~~~

From what we can understand, it appears that `rb_require_internal` calls `rb_feature_p` which ultimately calls `rb_file_expand_path_fast` and resizes a string.  It doesn't seem like the memory is ever freed either in c or the garbage collector.  This happens many times, perhaps because Pathname objects, unlike Strings, aren't cached in loaded features so they get expanded each time.   See below:

https://github.com/ruby/ruby/blob/v2_3_6/load.c#L43-L47

We can workaround this problem by converting Pathname objects in `$LOAD_PATH` to Strings, but this leak should be fixed since this it's common to use Pathname objects in `$LOAD_PATH`.

We used the instruments tool on OSX to show one such leak and the callstack, see attached image.

---Files--------------------------------
Instruments2 2018-01-18 11-43-32.png (181 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>