1

Preamble: This question is asked as a VM implementor, looking to support native callbacks (more specifically wndProc). For Process VMs I mean for Smalltalk, Java, Python, etc.

VMs are usually very careful of what gets into the stack. This is because during GC they will use the stack as a set of roots, and for each word in the stack they'll have to know whether it is a pointer to an object or not.

So here comes the problem: during a callback, the VM gets a bunch of stuff pushed into the stack. This may be just the arguments and return address, but might also contain much more things. Here is a little scheme, stack grows upwards:

oop <- managed
oop <- managed
-------------- <- native code end
... bunch of native stuff ...
-------------- <- native code start
oop <- managed
oop <- managed

From the point after the callback is entered, as the callback programmer one can control what gets into the stack again. But the question is, how do you control what happens before? Being more specific, how do you detect the native code start limit?

I'm guessing that the VM has a limit in the kind of callbacks, only allowing callbacks that will arrive when we call out, so the VM can save the limit of the stack at that moment. Is that correct? Is there any other alternative? I think using a separate stack could also be a solution, but I'm not sure of the performance implications.

Seki
  • 11,135
  • 7
  • 46
  • 70
melkyades
  • 1,719
  • 11
  • 20

1 Answers1

0

I think I answered myself. The main point is that callbacks cannot arrive at any time (i.e. asynchronously). The reason is the one below.

A callback is a simple user written C-style function that honors a calling convention. For example, in windows the CALLBACK type means stdcall. This means that the callback will only save some registers (the ones that are callee saved) and might overwrite others (the caller saved ones). If a callback arrived at any moment, then the running function wouldn't be prepared, and the caller saved registers could become corrupted. Then, the only moment when a callback is safe to be invoked is synchronously, after a callout has been done assuring a correct saving of context.

There might be other kind of callbacks (I'm thinking of interrupt routines and debugging code) that could be made asynchronous, but I think the main point is answered.

melkyades
  • 1,719
  • 11
  • 20