Issue #11101 has been updated by Thomas Kalmus.


I have looked at the thing closely and it seems that the GC is not CoW friendly for sparse arrays. If I create an array of undefined size and populate it with random numbers in string format, the GC will launch the CoW in the fork. However if the array has fixed size, or if I make a huge string of, for example, 30MB instead then I don't see the issue. Unfortunately, I am using a lot of dynamic arrays. Is there a way to bypass this ?

----------------------------------------
Bug #11101: Forking is killing my memory when running GC
https://bugs.ruby-lang.org/issues/11101#change-52457

* Author: Thomas Kalmus
* Status: Open
* Priority: Immediate
* Assignee: Thomas Kalmus
* ruby -v: 2.2.1
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------

Memory is duplicated when forking and calling GC in 2.2.1. The main issue here is that we are operating with huge data, going up to 3GB, and one fork is enough to kill our machine.

We have written a small program that reproduces the issue (see attached file).

    The program instantiates an object and then forks into two processes.
    The GC is called in the child process.
    The memory allocation (as given by /proc/pid/smaps) changes from shared to private, thereby indicating a doubling of memory consumption.

Here is the output of the program (size is in mb):

ruby version 2.1.3
   time   pid message             shared    private
 4.011s  4723 Parent pre GC           68          0
 4.013s  4737 Child  pre GC           68          0
 8.019s  4723 Parent post GC           5         62
 8.093s  4737 Child  post GC           5         66

We have tested the program on Ubuntu 14.04 with ruby 1.9.3 and 2.2.1. The tests have been performed on a freshly installed Ubuntu machine.

We have also tried to fork 10 children and see a 10 doubling of the memory consumption, the issue only occurs after running the GC.


---Files--------------------------------
mem.rb (1.25 KB)


-- 
https://bugs.ruby-lang.org/