2
def do_stuff_before_python_terminates():
    save_variables_in_mysql()
    do_this_and_that()...

def main():
    do stuff
    while loops ect...

def sigterm(x, y):
    raise Exception()
def sigint(signal, frame):
    raise Exception()





signal.signal(signal.SIGINT, sigint)
signal.signal(signal.SIGTERM, sigterm)  
try:
    while True:
        main()
except Exception as e:
    logger.error("Exception")
    do_stuff_before_python_terminates()
    logger.log("sys.exit")
    sys.exit(0)

I use Python in a Docker Container

When i do ctrl+c via terminal tty or stop the image python does stop and do it not always successfully "do_stuff_before_python_terminates()".

The reason is that when python is randomly in a while loop then i don't have luck it does not exit it stays in the while loop and do still other stuffs and do not terminate successfully.

Docker only waits 10 seconds before it kills the container and than voila it does not "do_stuff_before_python_terminates()"

what am i doing here wrong, how to solve the problem that even when its in a while loop it instantly exits and "do_stuff_before_python_terminates()"

Updated Explanation:

if python threws an error

it does successfully jump to 

except Exception as e:
    logger.error("Exception")
    do_stuff_before_python_terminates()
    logger.log("sys.exit")
    sys.exit(0)

if i stop the container or use ctrl+c and python is NOT in a while loop

it does successfully jump to

sigterm() or sigint() -> then raise an Exception() ->  then jumps to
except Exception as e:
    logger.error("Exception")
    do_stuff_before_python_terminates()
    logger.log("sys.exit")
    sys.exit(0)

if i stop the container or use ctrl+c and python is IN an Loop

it does stay in the loop 
do stuff
do stuff
do stuff
after nearly 20-30seconds
sigterm() or sigint() -> then raise an Exception() ->  then jumps to
except Exception as e:
    logger.error("Exception")
    do_stuff_before_python_terminates()
    logger.log("sys.exit")
    sys.exit(0)

what i need is, that it does instantly jumps out the while loop whenever i stop the container or do ctrl+c (sigterm + sigint)

sigterm() or sigint() -> then raise an Exception() ->  then jumps to
except Exception as e:
    logger.error("Exception")
    do_stuff_before_python_terminates()
    logger.log("sys.exit")
    sys.exit(0)

Docker does kill the container after 10seconds so, python only has 10 seconds to exit, the effect is that it does never do_stuff_before_python_terminates() when python is in a loop

Human Khoo
  • 135
  • 3
  • 9
  • 1
    Two possible explanations: code called from `main()` installs its own signal handler or code called from main catches the `Exception`. – VPfB Jul 13 '18 at 15:27
  • sorry no clue what you mean, i corrected the structure to be more readable maybe that was the reason it looked like signals are in main function , can you provide a solution – Human Khoo Jul 13 '18 at 16:12
  • I mean that some of the functions that are called may interfere with the exception raise and catch mechanism you are relying upon. Maybe it isn't the case, I cannot know, but you can test and rule it out by minimizing the code. – VPfB Jul 13 '18 at 16:36
  • 1
    Does the text `except:` or `except Exception` exist anywhere else in your code? Either of those might catch the signal-generated exception before it reaches the handler that does `do_stuff_before_python_terminates()`. – jasonharper Jul 13 '18 at 16:36
  • @jasonharper you mean that when python is in the while loop, there is in the while loop another try: ../ except: part that could catch the exception? i took a short look, i got 1k line of code and yes indeed that could be that in a while loop are some other try: except: parts, so i have to change these parts and do what exactly? i should add to all exceptions in my code something like if sigterm or sigint -> raise Exception ? what exactly is the exception to catch? i need to catch both sigint and sigterm. Did i understand it correctly, that stopping or using ctrl+c does throw an exception? – Human Khoo Jul 13 '18 at 16:44
  • 1
    Yes - if your code is currently executing inside another try/except, that gets first chance to catch the `Exception`. All of your `except`s need to use the most specific exception name possible, so that they don't catch things they shouldn't. – jasonharper Jul 13 '18 at 16:50
  • so the way is not, to add a exception for catching the sigint or sigterm, the way is that i don't use just except: , i should use specific except the one and only exception that can be thrown there? thats not easy possible because i don't even know which exceptions can get thrown there in what case scenario... is it not possible to add specific except sigint because than i can just raise Exception, that jumps to the do_stuff_before_python_terminates() if thats solving my issue – Human Khoo Jul 13 '18 at 17:03

3 Answers3

0

Your do_stuff_before_python_terminates() is not called properly (from my experience with signals and python at least).

Make the call to do_stuff_before_python_terminates() in your sigterm and/or sigint function.

def do_stuff_before_python_terminates():
    save_variables_in_mysql()
    do_this_and_that()...

def sigterm(x, y):
    do_stuff_before_python_terminates()
    raise Exception()
def sigint(signal, frame):
    do_stuff_before_python_terminates()
    raise Exception()

Make sure to put the do_stuff function above the sigterm & sigint functions.

J0hn
  • 570
  • 4
  • 19
  • 1
    This might be the proper way to write the signal handlers, but I would like to understand what's wrong with the original code. – VPfB Jul 13 '18 at 15:00
  • 1
    i added more information, thank you for your answer i appreciate that, but your answer does not help – Human Khoo Jul 13 '18 at 15:09
0

  1. Register the signal in main(). For example: 

        def signal_handler(signal, frame):
            print('\n')
            sys.exit(0)

    def main():
        signal.signal(signal.SIGINT, signal_handler)
        do_stuff_before_python_terminates()

  2. Signals won't fire until low-level C code finishes; from the documentation: "A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes."

gregory
  • 10,969
  • 2
  • 30
  • 42
  • what is the difference if i register them global before my code goes into main()? i only have the problem to jump out of while loops – Human Khoo Jul 13 '18 at 16:46
  • there is no "global" -- if you have a main(), the script starts there and proceeds from there. – gregory Jul 13 '18 at 16:57
  • my code does work from up to down, and then jumps in to main at the end of my code where i tell him to, outside of any function im registering the sigint and sigterm handler like in my example code – Human Khoo Jul 13 '18 at 17:04
  • Source of that quote: https://docs.python.org/3/library/signal.html – Max Barraclough Apr 30 '21 at 12:52
0

From Execution of Python signal handlers:

A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.

This may be the cause of your problem. Because of this limitation (and others), signal handlers written in Python rarely behave as expected except in the simplest programs.

If you really need to catch the signal immediately, you may need to write a wrapper for your program in a low level language (e.g. C) that calls your Python code.

  • thanks, but thats just not possible for me. i can't code in C and i need definitely python run do_stuff_before_python_terminates() on every sigterm or sigint (shutdown or ctr+l). – Human Khoo Jul 13 '18 at 17:13