Execution abruptly halting if the thread / process is killed makes sense
Why it won't execute cleanup code when I exit the main program normally by clicking the [X] on my terminal window?
I'm still learning the ins-and-outs of multithreaded applications, and I assume my problems come from not understanding how Python handles killing background threads.
Questions:
- Why won't my
finally:block execute all the time? - When else won't a
finally:block execute? - What happens to code execution inside a thread when the thread is killed?
- What happens to daemon/non-daemon threads when you exit the main process?
Details:
I'm trying to write a multithreaded program using ZMQ sockets that (among other things) writes stuff to a log file. I want the logging thread to unconditionally perform some messaging and clean-up right before it dies, but it won't most of the time.
The function below starts an infinite loop in a background thread and returns a zmq.PAIR socket for communication. The loop it starts listens to a socket, and anything written to that socket gets written to the file. The loop also (should) transmit back diagnostic messages like "I'm starting to log now!","Oops, there's been an error!" a "I'm exiting now". so the main program can keep tabs on it.
The main program generates a few threads using this pattern to monitor/control different bits and pieces. It polls several ZMQ sockets (connected to STDIN and a serial port) for messages, and forwards some of them to the socket connected to the file.
But now I'm stuck. The main program's routing & control logic works fine. get_logfile_sock's file writing works fine, and normal exception handling works as expected. But the "I'm exiting now" code doesn't execute when the thread is killed from the main program, or when I stop the main program altogether.
Example:
def get_logfile_sock(context, file_name):
"""
Returns a ZMQ socket. Anything written to the socket gets appended to the a specified file. The socket will send diagnostic messages about file opening/closing and any exceptions encountered.
"""
def log_file_loop(socket):
"""
Read characters from `socket` and write them to a file. Send back diagnostic and exception information.
"""
try:
socket.send("Starting Log File {}".format(file_name))
with open(file_name, "a+") as fh:
# File must start with a timestamp of when it was opened
fh.write('[{}]'.format(get_timestamp()))
# Write all strings/bytes to the file
while True:
message = socket.recv()
fh.write(message)
fh.flush()
# Un-comment this line to demonstrate that the except: and finally: blocks both get executed when there's an error in the loop
# raise SystemExit
except Exception as e:
# This works fine when/if there's an exception in the loop
socket.send("::".join(['FATALERROR', e.__class__.__name__, e.message]))
finally:
# This works fine if there's an exception raised in the loop
# Why doesn't this get executed when my program exits? Isn't that just the main program raising SystemExit?
# Additional cleanup code goes here
socket.send("Closing socket to log file {}".format(file_name))
socket.close()
# Make a socket pair for communication with the loop thread
basename = os.path.basename(file_name).replace(":", "").replace(" ", "_").replace(".", "")
SOCKNAME = 'inproc://logfile-{}'.format(basename)
writer = context.socket(zmq.PAIR)
reader = context.socket(zmq.PAIR)
writer.bind(SOCKNAME)
reader.connect(SOCKNAME)
# Start the loop function in a separate thread
thread = threading.Thread(target=log_file_loop, args=[writer])
thread.daemon = True # is this the right thing to do?
thread.start()
# Return a socket endpoint to the thread
return reader
