-1

I'm trying to find a way to use remote_pdb and entr together as an environment for rapid testing and development of Python 3 multiprocessing applications, but they're not playing nicely together.

Consider the trivial test case below:

import time
from multiprocessing import Process
from remote_pdb import RemotePdb
def child():
    RemotePdb('localhost', 4444).set_trace()
    while True:
        print("I am the child")
        time.sleep(1)
proc = Process(target=child)
proc.start()
proc.join()

If I save it in a file, say, 'mp.py' and the run it with:

python mp.py

it works very nicely. It breaks, as expected, in the child process displaying 

CRITICAL:root:RemotePdb session open at 127.0.0.1:4444, waiting for connection ...
RemotePdb session open at 127.0.0.1:4444, waiting for connection ...

and I can connect from another terminal session with, netcat or socat and use the python debugger. Exiting the debugger leaves the child and parent running until I explicitly kill them.

If I now comment out the RemotePdb() call and run the script under control of entr with 

ls mp.py | entr -r python mp.py

it allows me to edit mp.py and get an automatic clean restart each time I save.

The problem arises when I uncomment the RemotePdb() call.

The script is restarted by entr and it breaks in the child displaying the same 'waiting for connection' message as before but if I try to connect with netcat in another terminal window the pdb session is unresponsive with no output and no echo -- just empty lines in response to CR's. I get the same result with socat.

I'm developing on OS X 10.11.6 with python 3.5.3, entr 3.7, remote_pdb 1.2.

What I'm asking for: I'll accept an answer that provides a command line solution which combines the benefits of entr and remote_pdb or a clear explanation why it can't be done.

Thanks!

Update Further testing shows that the -r (restart) option to entr is necessary to produce the problem, i.e. 

ls mp.py | entr python mp.py

permits the debugger connection to work. Unfortunately, the restart option is essential to the workflow.

Update 2: The problem seems to be related to setting the process group. This is done by entr -r to make sure all the child processes can be killed. While trying to come up with a Python script to replace entr I found that calling os.setpgrp() in the parent process will produce the same failure (no I/O to debugger) if the script is launched from an IPython shell (!python mp.py) but not if it launched from bash.

Update 3: The author of remote_pdb and I have independently verified that the problem is not occurring in Linux. This is an OS X issue apparently having something to do with Python sockets and process groups. I've added to the tags accordingly.

Mike Ellis
  • 1,120
  • 1
  • 12
  • 27

1 Answers1

0

Solved! It's an old OS X python issue that has either crept back in or was never fixed in Pdb. The fix is to set up to ignore SIGTTOU before importing readline. See http://bugs.python.org/issue14892 for details.

I cured the problem by patching my local copy of pdb.Pdb as shown below.

class Pdb(bdb.Bdb, cmd.Cmd):

    _previous_sigint_handler = None

    def __init__(self, completekey='tab', stdin=None, stdout=None, skip=None,
                 nosigint=False):
        bdb.Bdb.__init__(self, skip=skip)
        cmd.Cmd.__init__(self, completekey, stdin, stdout)
        if stdout:
            self.use_rawinput = 0
        self.prompt = '(Pdb) '
        self.aliases = {}
        self.displaying = {}
        self.mainpyfile = ''
        self._wait_for_mainpyfile = False
        self.tb_lineno = {}
        # Try to load readline if it exists
        try:
############ FIX OSX BUG ######################################
            import sys
            if sys.platform == 'darwin':
                import signal
                signal.signal(signal.SIGTTOU, signal.SIG_IGN)
############ END FIX ##########################################
            import readline
            # remove some common file name delimiters
            readline.set_completer_delims(' \t\n`@#$%^&*()=+[{]}\\|;:\'",<>?')
        except ImportError:
            pass

Many thanks to ionelmc, author of remote_pdb and hunter for generous help tracking down the problem.

Mike Ellis
  • 1,120
  • 1
  • 12
  • 27