Issue #14718 has been updated by bluz71 (Dennis B).


I have taken Mike's script, increased the `THREAD_COUNT` to 20 and run it on my Linux box (Intel i5-4590 quad-core, 16GB RAM, Linux Mint 18.3 with kernel 4.15.0).

glibc malloc results:

~~~
% time MALLOC_ARENA_MAX=1 ruby frag.rb
VmRSS:   4,164,108 kB
real     12.001s

% time MALLOC_ARENA_MAX=2 ruby frag.rb
VmRSS:   4,362,360 kB
real     12.259s

% time MALLOC_ARENA_MAX=3 ruby frag.rb
VmRSS:   4,486,620 kB
real     12.618s

% time MALLOC_ARENA_MAX=4 ruby frag.rb
VmRSS:   4,692,404 kB
real     12.064s

% time MALLOC_ARENA_MAX=5 ruby frag.rb
VmRSS:   4,682,908 kB
real     12.364s

% time MALLOC_ARENA_MAX=6 ruby frag.rb
VmRSS:   4,914,360 kB
real     12.249s

% time MALLOC_ARENA_MAX=7 ruby frag.rb
VmRSS:   5,063,128 kB
real     12.507s

% time MALLOC_ARENA_MAX=8 ruby frag.rb
VmRSS:   5,325,748 kB
real     12.598s

% time MALLOC_ARENA_MAX=12 ruby frag.rb
VmRSS:   5,965,244 kB
real     12.321s

% time MALLOC_ARENA_MAX=16 ruby frag.rb
VmRSS:   6,565,000 kB
real     11.827s

% time MALLOC_ARENA_MAX=24 ruby frag.rb
VmRSS:   7,106,460 kB
real     12.007s

% time MALLOC_ARENA_MAX=32 ruby frag.rb
VmRSS:   7,101,580 kB
real     11.646s

% time ruby frag.rb
VmRSS:   7,241,108 kB
real     11.739s
~~~

And with jemalloc 3.6.0 (1st stanza) and 5.0.1 (2nd stanza):

~~~
% time LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so ruby frag.rb
VmRSS:   4,680,260 kB
real     25.103s

% time LD_PRELOAD=/home/bluz71/.linuxbrew/Cellar/jemalloc/5.0.1/lib/libjemalloc.so ruby frag.rb
VmRSS:   7,162,508 kB
real     12.111s
~~~

I am not sure if this test is really showing memory fragmentation or arena count expense.

Mike's test clearly runs twice as slow via jemalloc 3.6.0. Conversely when run with jemalloc 5.0.1 the performance issue goes away but the `VmRSS` size matches high arena count glibc malloc.

I do use jemalloc 3.6.0 with my Rails application with great results. Noah Gibbs noted excellent results when using jemalloc with his Rails benchmark tests as noted [here](http://engineering.appfolio.com/appfolio-engineering/2018/2/1/benchmarking-rubys-heap-malloc-tcmalloc-jemalloc).

Nevertheless we should indeed be cautious. I tend to agree now with backing off using jemalloc by default since different versions have wildly different results with Mike's test.

----------------------------------------
Feature #14718: Use jemalloc by default?
https://bugs.ruby-lang.org/issues/14718#change-72145

* Author: mperham (Mike Perham)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
I know Sam opened #9113 4 years ago to suggest this but I'm revisiting the topic to see if there's any movement here for Ruby 2.6 or 2.7.  I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps.  My users switch to jemalloc and a miracle occurs: their memory usage drops massively.  Some data points:

https://twitter.com/brandonhilkert/status/987400365627801601
https://twitter.com/d_jones/status/989866391787335680
https://github.com/mperham/sidekiq/issues/3824#issuecomment-383072469

Redis moved to jemalloc many years ago and it solved all of their memory issues too.  Their conclusion: the glibc allocator "sucks really really hard". http://oldblog.antirez.com/post/everything-about-redis-24.html

This is a real pain point for the entire Rails community and would improve Ruby's reputation immensely if we can solve this problem.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>