Python pdb breakpoint after 10 seconds

Question

On some occasions, my python program won't response because there seems to be a deadlock. Since I have no idea where this deadlock happens, I'd like to set a breakpoint or dump the stack of all threads after 10 seconds in order to learn what my program is waiting for.

Just Ctrl-C your program. This will raise a `KeyboardInterupt` which will print a stack trace showing where the execution was when the exception was raised. — Dunes, Aug 04 '14 at 21:45
@Dunes I already tried that but this exception seems to be caught and ignored. — Matt3o12, Aug 04 '14 at 21:48
You could spawn a thread, then sleep for 10 seconds, but when you `pdb` from there you won't be in the same context where the failure is happening. It may help though, depending on what you're looking for — Matt Dodge, Aug 04 '14 at 21:53
It is a very bad thing to be unintentionally catching a keyboard interrupt. It means you are catching all sorts of exceptions that you shouldn't be like `SystemExit` and `MemoryError`. Search your code for statements like `except:` or `except BaseException:`. — Dunes, Aug 04 '14 at 21:54
@Dunes I've found out that my code isn't catching any of these exception. I think it is either nose (that is running the testcase) or psycopg2 but I'm not sure. — Matt3o12, Aug 04 '14 at 21:58
Nose should be taking a KeyboardInterrupt as a sign to abort the tests. Not sure what's going on... — Dunes, Aug 04 '14 at 22:16

score 0 · Answer 1 · answered Aug 04 '14 at 22:05

Use the logging module and put e.g. Logger.debug() calls in strategic places through your program. You can disable these messages by one single setting (Logger.setLevel) if you want to. And you can choose if you want to write them to e.g. stderr or to a file.

score 0 · Answer 2 · answered Aug 04 '14 at 22:15

0

import pdb
from your_test_module import TestCase

testcase = TestCase()
testcase.setUp()
pdb.runcall(testcase.specific_test)

And then ctrl-c at your leisure. The KeyboardInterupt will cause pdb to drop into debugger prompt.

answered Aug 04 '14 at 22:15

Dunes

37,291
7
81
97

score 0 · Accepted Answer · edited Aug 05 '14 at 15:36

Well, as it turns out, it was because my database was locked (a connection wasn't closed) and when the tests were tearing down (and the database schema was being erased so that the database is clean for the next tests), psycopg2 just ignored the KeyboardInterrupt exception.
I solved me problem using the faulthandler module (for earlier versions, there is a pypi repo). Fault handler allows me to dump the stack trace to any file (including sys.stderr) after a period of time (repeatingly) using faulthandler.dump_traceback_later(3, repeat=True). That allowed me to set the breakpoint where my program stopped responding and tackle the issue down effectively.

Python pdb breakpoint after 10 seconds

3 Answers3