1

I get this strange behaviour (random TypeError exception is thrown) whenever I interrupt/kill a sleeping parent thread from a child thread in python 3 but the strange behaviour ONLY appears if the python script was initiated by a shell script with an & argument to make it run in the background.

So here's the minimum reproducible python code I could come up with that can trigger this issue.

User@MSI: ~/test $ cat m.py

import threading
import _thread
import time


def child_thread():
    time.sleep(1)
    print('child interrupting parent')
    _thread.interrupt_main()
    
if __name__ == '__main__':
    t = threading.Thread(target=child_thread, args=())
    t.start()
    print('parent looping')
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        print('caught interruption raised from user or child thread :)')
    except TypeError as e:
        print('why would I ever catch a TypeError?')
        raise
    except Exception as e:
        print('Strange Exception Received ')
        raise

It's a very simple script where a parent initiates a child thread and then loops forever and the child thread (after 1 second) interrupts the main thread which SHOULD raise a KeyboardInterruption towards the main thread.

Now if I invoke my script using a bash script with the background flag & at the end, (the below output makes no sense)

User@MSI: ~/test $ cat ./m.sh
#!/bin/bash
python3 m.py &

User@MSI: ~/test $ ./m.sh
parent looping
child interrupting parent
why would I ever catch a TypeError?
Traceback (most recent call last):
  File "m.py", line 17, in <module>
    time.sleep(1)
TypeError: 'int' object is not callable

below I show the output of my script behaving completely normal when I run it from

  1. From the terminal
  2. From the terminal with &
  3. From the bash script (without &)

And all three behave as expected... (the below output is as expected)

User@MSI: ~/test $ python3 m.py
parent looping
child interrupting parent
caught interruption raised from user or child thread :)

Same result when running it in the background with & (the below output is as expected)

User@MSI: ~/test $ python3 m.py &
[1] 5884
parent looping
child interrupting parent
caught interruption raised from user or child thread :)

[1]+  Done   python3 m.py

And even from a one line script called m.sh that executes m.py (the below output is as expected)

User@MSI: ~/test $ cat m.sh
#!/bin/bash
python3 m.py

User@MSI: ~/test $ ./m.sh
parent looping
child interrupting parent
caught interruption raised from user or child thread :)

I'm completely dumbfounded and have no idea what time.sleep and TypeErrors have to with how I invoked my script and specifically invoking it from a shell script and from the backgrounded. This is one of the strangest errors I've encountered.

If it matters, I'm running Python 3.6.12 and Debian 9.12

I hope someone can figure this out.

EDIT: Here are comparisons of the bytecode for the buggy version's (run from shell script with &) output And here's the good version's (run from terminal) output

And for easy comparison, here's a diff of the bytecode. The only difference is the location of the child thread in memory.

A Kareem
  • 586
  • 3
  • 10
  • Cannot reproduce. The error message suggests you have assigned some other value to the name `time`, but that doesn't occur in the code shown. – chepner Dec 24 '20 at 16:05
  • Is there a python file named "time.py" in that directory? – Revisto Dec 24 '20 at 16:09
  • @chepner That's what it suggests (the only ever mention of this I found online was someone actually assigning an int to time.sleep), but the code I posted was an exact copy to the code I'm running. I've also restarted my machine several times and tried several different variations of the script, in fact, I initially faced the issue in a much bigger project and it was very odd. Although I'm very curious what behaviour do you get when you run it in a bash script with &? Do you actually get the correct print line with a smiley face??? – A Kareem Dec 24 '20 at 16:09
  • @Revisto no, the only two files are m.py and m.sh the directory was meant only to reproduce this bug in a contained environment for the sake of posting this question and make it as clear as possible. – A Kareem Dec 24 '20 at 16:10
  • This sounds like the failure case was running different Python code with a typo like `time.sleep=(1)` in the child thread. If you have an `m.pyc` file, delete it. – user2357112 Dec 24 '20 at 16:20
  • @user2357112supportsMonica Unfortunately there are no `.pyc` files. Also, just for the sake of triple checking 100%. I created and went to a completely new directory, created two files `a.py` and `b.sh` and copy pasted the code that is in the question above (except obviously changed m.py to a.py) then ran chmod +x on `b.sh` and then ran the command `./b.sh` and I got the same TypeError again. Then ran `python3 a.py` and it ran smoothly just as above. Very strange. – A Kareem Dec 24 '20 at 16:26
  • 1
    Try checking `__file__` and `sys.executable` inside the script, and see if they're what you expect. Also `print(time.sleep)` inside the exception handler, and see if it somehow got replaced with an int. – user2357112 Dec 24 '20 at 16:30
  • 1
    You may also want to try importing `dis` and running `dis.dis(child_thread)` and `dis.dis(sys._getframe().f_code)`, to inspect the actual bytecode Python is using and see if it matches the script you think you should be running. – user2357112 Dec 24 '20 at 16:32
  • 1
    (Something like [this](https://pastebin.com/FHeiCjKj).) – user2357112 Dec 24 '20 at 16:34
  • @user2357112supportsMonica Good debugging although, `sleep.time` is still a built in function. and `__file__` is just as expected and byte code looks... fine as far as I can tell? (although I'm not at all good with low level instructions) Here's the buggy version's (shell with &) [output](https://pastebin.com/9TCfNFAY) And here's the good version's (terminal) [output](https://pastebin.com/4GtH5VFA) – A Kareem Dec 24 '20 at 16:46
  • 1
    Tried reproducing the bug, on Python 3.8.0. Got a different symptom - instead of a TypeError, the main thread just isn't interrupted at all. There might be an actual Python bug here. – user2357112 Dec 24 '20 at 17:05
  • Good to know that I wasn't going crazy, it might possible be, I'll make sure to submit it as an official bug report. Thanks a lot @user2357112supportsMonica – A Kareem Dec 24 '20 at 17:10
  • 1
    I found a [related bug report](https://bugs.python.org/issue23395). It looks like the behavior we're seeing indicates that the signal handling settings for SIGINT have been changed to ignore it - I'm not sure why. The behavior you're seeing is an old bug where `interrupt_main` wasn't handling the "ignore" setting correctly, while the behavior I'm seeing is just the simulated SIGINT being ignored. – user2357112 Dec 24 '20 at 17:24
  • Oh no, I just posted my bug report after searching for a bit and not finding any previous bug reports related to `time.sleep` and `TypeError`, I should've definitely searched for `_thread.interrupt_main` as well. – A Kareem Dec 24 '20 at 17:33

0 Answers0