Okay, I did a little bit more digging into ruby's source code (both 1.8
and 1.9), and it seems the problem lies in Init_stack() (and its cousin
ruby_init_stack in later ruby versions) within gc.c.

Currently, Init_stack() does not guarantee that the stack is reset.
Ruby seems to do a bounds check first and only then decides whether to
reset its internal variable (rb_gc_stack_start).

This can easily lead ruby to think the stack is about to overflow and
raise a stack error on an embedded ruby where ruby functions are called
from drastically different addresses in memory.

Unfortunately, for an embedded application, this is problematic.  You
really want ruby_init_stack() to init the stack to the address passed
and not do the bounds check.  Or is that the bounds check is being
incorrectly done (reversed) on x86-64 AMD machines?

When I remove the bounds check, all works as expected.

Can anyone explain the logic of the current behavior?

---

Here's a sample run of what happens with an embedded ruby where some
methods are being called as callbacks from C++:

>>> Embedded ruby is inited (and some basic config stuff is run)
ruby_init_stack: (nil) at START
ruby_init_stack: should reset to 0x7fffb12eb980
ruby_init_stack: 0x7fffb12eb980 at END
Init_stack: 0x7fffb12eb980 at START
Init_stack should reset to: 0x7fffb12eb964
Init_stack: 0x7fffb12eb980 at END
mark stack 0x7fffb12eb980 - 0x7fffb12e7820- MARKED

Starting Rendering /home/gga/maya/projects/default/images/untitled.iff.
Rendering current frame.

>>>
>>> Render begins
>>> shader's callback is called, ruby_init_stack(&localvar) is used
>>> to reset stack, but it fails to do so.
>>>
ruby_init_stack: 0x7fffb12eb980 at START
ruby_init_stack: should reset to 0x418021d0
ruby_init_stack: 0x7fffb12eb980 at END   -- **NOT RESET**

ruby_init_stack: 0x7fffb12eb980 at START
ruby_init_stack: should reset to 0x418021d0
ruby_init_stack: 0x7fffb12eb980 at END   -- **NOT RESET**

Error: [Ruby] - /usr/local/lib/ruby/1.8/complex.rb:107:in `generic?'
....etc....


-- 
Gonzalo Garramu?o
ggarra / advancedsl.com.ar

AMD4400 - ASUS48N-E
GeForce7300GT
Kubuntu Edgy