Does finally ensure some code gets run atomically, no matter what?

Question

Assume I'm going to write a Python script that catches the KeyboardInterrupt exception to be able to get terminated by the user using Ctrl+C safely

However, I can't put all critical actions (like file writes) into the catch block because it relies on local variables and to make sure a subsequent Ctrl+C does not break it anyway.

Would it work and be good practice to use a try-catch block with empty (pass) try part and all the code inside the finally part to define this snippet as "atomic, interrupt-safe code" which may not get interrupted mid-way?

Example:

try:
    with open("file.txt", "w") as f:
        for i in range(1000000):
            # imagine something useful that takes very long instead
            data = str(data ** (data ** data))
            try:
                pass
            finally:
                # ensure that this code is not interrupted to prevent file corruption:
                f.write(data)

except KeyboardInterrupt:
        print("User aborted, data created so far saved in file.txt")
        exit(0)

In this example I don't care for the currently produced data string, i.e. that creation could be interrupted and no write would be triggered. But once the write started, it must be completed, that's all I want to ensure. Also, what would happen if an exception (or KeyboardInterrupt) happened while performing the write inside the finally clause?

`finally` does not mean "atomic, interrupt-safe code". All sorts of things can stop a `finally` or prevent it from running, including interrupts. It's just a control flow statement. — user2357112, Apr 21 '16 at 06:42

Martijn Pieters · Accepted Answer · 2016-04-21T07:25:59.710

5

Code in finally can still be interrupted too. Python makes no guarantees about this; all it guarantees is that execution will switch to the finally suite after the try suite completed or if an exception in the try suite was raised. A try can only handle exceptions raised within its scope, not outside of it, and finally is outside of that scope.

As such there is no point in using try on a pass statement. The pass is a no-op, it won't ever be interrupted, but the finally suite can easily be interrupted still.

You'll need to pick a different technique. You could write to a separate file and move that into place on successful completion; the OS guarantees that a file move is atomic, for example. Or record your last successful write position, and truncate the file to that point if a next write is interrupted. Or write markers in your file that signal a successful record, so that reads know what to ignore.

edited Apr 21 '16 at 07:25

answered Apr 21 '16 at 06:24

Martijn Pieters

1,048,767
296
4,058
3,343

Yes, sorry. This was just a hypothetical example and I just noticed that it would not have been useful this way myself. I updated the example. – Byte Commander Apr 21 '16 at 06:27
@ByteCommander: same comments apply. The noop won't be interrupted, your `finally` suite is not going to be executed. – Martijn Pieters Apr 21 '16 at 06:33
1

@ByteCommander: you also seem to think that `data` will receive a value as `str(..)` produces data. That's not the case. A Python expression has to *run to completion* before assignment can take place. If the `str(..)` call is interrupted, `data` will never be assigned to. – Martijn Pieters Apr 21 '16 at 06:37
Yes, of course. I said that I don't care about the current data snippet, I just want to make sure the file write action gets completed at once without being interrupted. If an interrupt happens before a write starts, it's okay to discard the not yet saved data produced so far. – Byte Commander Apr 21 '16 at 06:55
@ByteCommander: right, that wasn't clear in the first iteration of your question. I've updated your title to ask what you are really asking. – Martijn Pieters Apr 21 '16 at 07:20
Thanks for the edit, now we're talking about the same thing. :) – Byte Commander Apr 21 '16 at 07:26
@MartijnPieters Would you mind explaining the following? "The noop won't be interrupted, your finally suite is not going to be executed." My understanding is that the finally block is executed regardless of what happens in the try block (exception or no). And a `try: pass \nfinally: print("finally")` seems to contradict you. – Dunes Apr 21 '16 at 07:34
@Dunes: yes, it is executed, but there is no point in using it on `pass`. The only reason to use `finally` is to ensure stuff runs after other stuff, regardless of why the other stuff exited. `pass` will never exit. – Martijn Pieters Apr 21 '16 at 08:31
@Dunes: I didn't say `finally` wouldn't get executed. I'm saying the `try...finally` is redundant. – Martijn Pieters Apr 21 '16 at 08:32

score 5 · Answer 2 · edited Apr 21 '16 at 07:19

5

In your case, there is no problem, because file writes are atomic, but if you have some file object implementetion, that is more complex, your try-except is in the wrong place. You have to place exception handling around the write:

try:
    f.write(data)
except:
    #do some action to restore file integrity
    raise

For example, if you write binary data, you could to the following:

filepos = f.tell()
try:
    f.write(data)
except:
    # remove the already written data
    f.seek(filepos)
    f.truncate()
    raise

edited Apr 21 '16 at 07:19

Martijn Pieters

1,048,767
296
4,058
3,343

answered Apr 21 '16 at 06:34

Daniel

42,087
4
55
81

So one file write is atomic, but multiple subsequent file writes would not be, which means I would have to group them somehow to ensure they're all made. Is there no way to ensure a code block gets properly finished once it got started? Do I only have the option to roll back partial, incomplete actions? – Byte Commander Apr 21 '16 at 06:58
@Daniel: yes, but I was thrown by the title. The edit made things a little clearer. – Martijn Pieters Apr 21 '16 at 07:09
1

"file writes are atomic" - that's a misleading statement. Sufficiently up-to-date versions of Python 2.7 or Python 3.x will keep writing if interrupted by a signal, but anyone stuck on an older version can have their `file.write` calls interrupted. There are also other problems that could interfere with a file write, like someone jostling a USB stick at the wrong time. – user2357112 Apr 21 '16 at 07:21
@user2357112: as I stated, "... in your case .. "; there are many other failures, that are outside the scope of pythons exception handling. – Daniel Apr 21 '16 at 08:25

Does finally ensure some code gets run atomically, no matter what?

2 Answers2