On Thu, Apr 7, 2011 at 6:18 PM, Eric Wong <normalperson / yhbt.net> wrote:
> Evan Phoenix <evan / fallingsnow.net> wrote:
>> Has there been any thought on solving the C extension problem in MVM?
>> In the present state, I've stopped working on it in Rubinius because
>> there is no workable solution if there are C extensions in the mix.
>
> Is it just avoiding redundant load of dynamic libs and remembering to
> call Init_foo() on a per-VM basis?
>
> I suppose the other problem is C-level global variables and
> rb_global_variable(), but not all extensions use those...
>
> I think a useful subset of C extensions can be made to work with MVM,
> but I haven't been thinking of this problem for long (only since reading
> your email :)
>
> Perhaps you or someone can elaborate or point me to a writeup on the
> problem(s).

Almost all extensions store classes, symbols, constant values, and
more in C globals. If they do that, they can't be made safe with MVM,
since those globals all indirectly reference VM-specific data. And
since we can't determine that they do this on load, ko1 originally
thought to introduce a new Init entry point that's a "promise" by the
extension that it doesn't do MVM-unsafe things. I believe they had a
few extensions working that way when the primary MVM dev push tailed
off.

Of course we haven't even started to discuss concurrency with C
extensions, which is damn near impossible to do safely.

In JRuby, where we've had MVM support forever, C extensions will error
out almost immediately if loaded into a second VM. Different VMs can
load different extensions, but the same extension loaded twice will
not work.

We also have a global lock around extension calls, so only one can be
executing at a given time.

I strongly believe Ruby needs to have its C API overhauled, or at
least someone needs to seriously explore what APIs can be safe with
MVM and which APIs need fixing/replacing.

FWIW, none of this applies to JRuby extensions written in Java; they
are bound per VM and do not use any "global" state.

- Charlie