How can I trap a stack smashing from another thread? Is it possible to set up and clear a watchpoint on the fly to watch the stack canary?
Here is the code we are working with:
void SomeClass::func() {
fA();
fB();
}
void SomeClass::threadFunc() {
while(true) {
eventQueue.wait();
func();
}
}
The fB() function calls fC(), fD(), fE(), fF(), etc. The program sometimes
crashes with a "stack smashing" error in fE(), which is caused by the GCC
-fstack-protector option.
However, fE() is simple and I cannot see a way for the stack canary to be
overwritten.
Things get more interesting when we modify the code as follows:
void SomeClass::func() {
// lambda expression lA.
[&]{
uint8_t padA[nA];
fA();
}();
// lambda expression lB.
[&]{
uint8_t padB[nB];
fB();
}();
}
By adjusting the size of the padA and padB arrays, we can adjust the size of the
stack frame of lambda expression A and B. We can make their "top" the
same (if we say the stack grows "up"). If we do this, the program crashes due to
the canary of fE() being corrupted, as before.
By adjusting the size of padA and padB, we can make the program crash at
fC(), fD(), or fF() if the positions match the same point relative to the
stack frame of fA().
This implies that the stack of fB() is corrupted at a position determined by the
position of the stack frame of fA(). My theory is that fA() or any functions
called by it captured the address of some local variable and handed it over to
another thread. This thread has a chance to corrupt the stack when the timing is
wrong.
I have tried using GDB to set a non-stop breakpoint, which in turn sets a watchpoint on the canary. However, the program must interact with IO in real-time, and the performance of the GDB breakpoint is too low to let the program continue.
Is there a way to program the debug registers DR0 and DR7 on the fly to trap the unexpected access of the stack canary? This program runs on Linux and x86_64.
