Detect recursion robustly even in the presence of non-local jumps

Question

I have a particular function (a signal handler) for which I'd like to detect recursion, i.e. to figure out if the function has directly or indirectly called itself. The tricky bit is that the function calls some code not under its control at one point, and that code could do anything.

Normally, I'd just write something like

void foo() {
    static int recursed = 0;
    if(recursed) {
        ...
    }
    recursed = 1;
    othercode();
    recursed = 0;
}

but in this case, I'm concerned that othercode could use a longjmp or similar to break out, resulting in recursed remaining at 1. In the event that my function is jumped out of in this way, I want to ensure it doesn't see itself as recursing if is called later (the fact that it is longjmp'd out of is not a problem otherwise).

Note: I consider longjmp to be likely. The othercode is a chained signal handler from some other in-the-wild code, and there do exist handlers for e.g. SIGSEGV which use longjmp to restore context (e.g. as "fault protection" exception handlers). Note that using longjmp in a synchronous signal handler is generally safe. In any case, I don't particularly care whether the other code is safe at all, because it's not something under my control.

If you know where `jongjmp` goes, you can put code there to clear the recursion indication. — ugoren, Jun 05 '13 at 10:46

score 3 · Answer 1 · answered Jun 05 '13 at 04:58

3

Not sure what exactly the code would look like to do it, but instead of a static int, you could have a static void *. Instead of setting it to 1, set it to point to the current stack frame. In addition to checking if it's nonzero, you check to make sure that the return address from the next stack frame after recursed in fact points to a location in foo's code, and also that recursed is above the current stack pointer, i.e. not popped.

Sounds very fragile and architecture-dependent.

answered Jun 05 '13 at 04:58

morningstar

8,952
6
31
42

Or maybe it's simpler to look at the return address from foo's stack frame points right after a call instruction to foo, but that would *definitely* be non-portable. – morningstar Jun 05 '13 at 05:01
I considered this. It works really well, except for one tiny corner case: suppose `othercode` jumps out to a function well above `foo`, which calls some other function that allocates (but doesn't completely use) a giant stack buffer, then ends up triggering `foo`. If I'm unlucky, the giant stack buffer both moves the stack pointer below `foo` and preserves the original `foo` stack, making it look very much like a recursion. – nneonneo Jun 05 '13 at 06:37
Of course, this is rather unlikely, which is why I will probably end up using this solution. But, I would ideally like to find a solution that does not have this problem. – nneonneo Jun 05 '13 at 06:37
Maybe I'm thinking of the base / frame pointer rather than the stack pointer. – morningstar Jun 05 '13 at 14:45
Never mind, same problem. The allocated but not initialized buffer could be in a previous frame. – morningstar Jun 05 '13 at 16:37

score 0 · Answer 2 · answered Jun 05 '13 at 05:13

According to the POSIX standard for signal(7), longjmp() is not one of the calls that are safe to call from inside of a signal handler. Before even thinking about this issue the longjmp(3) documentation says, you need to make sure the code you are calling uses sigsetjmp() and siglongjmp()

If the code you are calling is jumping out of the signal handler, then I don't see how you can know when to update your recursed variable, unless you also control a callback function that this unknown code calls back into your application.

These only apply to **async signals**. I am writing a signal handler for `SIGSEGV` and other normally-synchronous signals, so this isn't expected to be an issue. — nneonneo, Jun 05 '13 at 06:28

Detect recursion robustly even in the presence of non-local jumps

2 Answers2