Issue #14718 has been updated by bluz71 (Dennis B).


mperham (Mike Perham) wrote:
> ~~~
> Bundle jemalloc and link it statically.
> 
> This would be my choice as a maintainer and I trust antirez knows what he's doing when he made the same choice for Redis.  You control the exact version and any resulting bugs.  If you depend on the distro's package, you will integrate with a variety of versions, all with their own set of platform-specific bugs and crash reports.  Ubuntu 14 might have jemalloc 3.6.0, 16 might have 4.2.0, 18 might have 5.0.1, etc.  Dealing with those platform issues will also cost the core devs.

This is a very strong point.

Legacy versions of jemalloc are solid as a rock. Newer versions have a higher chance of unexpected quirks. For instance Sam Saffron had unusual results with jemalloc 4.3.0:

https://github.com/SamSaffron/allocator_bench

Redis ships jemalloc 4.0.3 (or near to) as seen here:

https://github.com/antirez/redis/tree/unstable/deps/jemalloc

If Ruby was to ship jemalloc (also my preference), why not just take the same version as Redis? It has been battle-tested for seven years. We could also chat with Sam Saffron and the jemalloc maintainers for their thoughts about a preferred jemalloc version for Ruby. Ruby could ship and use that jemalloc and lock it in at a known and stable version.

> Linux/glibc is the platform for 90% of Ruby users in production.  It's ok to support FreeBSD and OpenBSD but they are very small in terms of actual usage.  If you want to make things better for the vast majority of the community, focus on Linux.

The FreeBSD system allocator already is jemalloc based. I am lead to believe that was thee first usage of Jason Evans memory allocator.

Replacing the OpenBSD memory allocator has been explained as an anti-pattern by Jeremy a few posts up. I would not replace it.

I tend to agree now with limiting any jemalloc changes just to the Linux platform where the memory fragmentation problem exists.

I'd still keep the `--with-jemalloc` option; this would always build in an externally provided jemalloc library (even on Linux, e.g a Linux or FreeBSD user wishing to use jemalloc 5.1.0). A new `--without-jemalloc` option would be nice to provide which would be a no-op on most platforms and on Linux it would force usage of glibc malloc (when desired).

Lastly, Ruby would not be on the bleeding edge if it adopted jemalloc (on Linux), this allocator is already used by default in the following:

- Redis database
- MarkLogic database
- Firefox browser
- Rust language
- Facebook

P.S. I have no links with jemalloc, but I do use it now with Ruby and I did successfully for many years in my previous database server life.

----------------------------------------
Feature #14718: Use jemalloc by default?
https://bugs.ruby-lang.org/issues/14718#change-71971

* Author: mperham (Mike Perham)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
I know Sam opened #9113 4 years ago to suggest this but I'm revisiting the topic to see if there's any movement here for Ruby 2.6 or 2.7.  I supply a major piece of Ruby infrastructure (Sidekiq) and I keep hearing over and over how Ruby is terrible with memory, a huge memory hog with their Rails apps.  My users switch to jemalloc and a miracle occurs: their memory usage drops massively.  Some data points:

https://twitter.com/brandonhilkert/status/987400365627801601
https://twitter.com/d_jones/status/989866391787335680
https://github.com/mperham/sidekiq/issues/3824#issuecomment-383072469

Redis moved to jemalloc many years ago and it solved all of their memory issues too.  Their conclusion: the glibc allocator "sucks really really hard". http://oldblog.antirez.com/post/everything-about-redis-24.html

This is a real pain point for the entire Rails community and would improve Ruby's reputation immensely if we can solve this problem.



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request / ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>