-1

I'd like to make a program which saves intermediate results to be able to reuse them is case of crash.

In this regard, I wanted to try to open the intermediate results file with a context manager. However basic context manager do not include error handling and basic error handling do not include context management. So I was forced to write my own context manager - which works fairly well, but I do not feel like the result is very pythonicly satifying

import subprocess
import contextlib
import sys

@contextlib.contextmanager
def myopen(cmd, filename):
    try: f = open(filename, 'r')
    except FileNotFoundError: pass
        # I didn't put the file creation here to prevent the risk of
        # causing an exception during the handling of the previous one 
    else:
        yield f
        f.close()
        return

    print(f'creating new {filename}', file=sys.stderr, flush=True)
    sp = subprocess.run(cmd, shell=True, executable='/bin/bash')
        
    try: f = open(filename, 'r')
    except: raise
    else:
        yield f
        f.close()

Which I test with

filename = "./context_exception.data"

print(f'removing {filename}', file=sys.stderr, flush=True)
sp = subprocess.run("rm "+filename, shell=True, executable='/bin/bash')

cmd = "echo -e 'spam\\neggs' >"+filename
with myopen(cmd, filename) as f:
    lines=f.readlines()
    for i, line in enumerate(lines):
        print(f'line{i:02}={line}', end='')

cmd = "echo -e 'ham\\nbanana' >"+filename
with myopen(cmd, filename) as f:
    lines=f.readlines()
    for i, line in enumerate(lines):
        print(f'line{i:02}={line}', end='')

print(f'removing {filename}', file=sys.stderr, flush=True)
sp = subprocess.run("rm "+filename, shell=True, executable='/bin/bash')

with myopen(cmd, filename) as f:
    lines=f.readlines()
    for i, line in enumerate(lines):
        print(f'line{i:02}={line}', end='')

Is there any nicer way to combine context manager and exception handling, without having to manually rewrite the context manager ? And by the way, I'm not very satisfied of the bulkiness of my context manager. Any idea about how to make it more compact would be welcome.

EDIT (Addendum)

Obviously, as @PeterWood noticed, I could, just test the existence beforehand, and then wouldn't even need neither to rewrite a context manager nor any exception handling:

import subprocess
import os
import sys


def create_if_nonexistent(cmd, filename):
    if not os.path.isfile(filename):
        print(f'creating new {filename}', file=sys.stderr, flush=True)
        sp = subprocess.run(cmd, shell=True, executable='/bin/bash')


filename = "./context_exception.data"
print(f'removing {filename}', file=sys.stderr, flush=True)
sp = subprocess.run("rm "+filename, shell=True, executable='/bin/bash')


cmd = "echo -e 'spam\\neggs' >"+filename
create_if_nonexistent(cmd, filename)
with open(filename) as f:
    lines=f.readlines()
    for i, line in enumerate(lines):
        print(f'line{i:02}={line}', end='')


cmd = "echo -e 'ham\\nbanana' >"+filename
create_if_nonexistent(cmd, filename)
with open(filename) as f:
    lines=f.readlines()
    for i, line in enumerate(lines):
        print(f'line{i:02}={line}', end='')


print(f'removing {filename}', file=sys.stderr, flush=True)
sp = subprocess.run("rm "+filename, shell=True, executable='/bin/bash')


create_if_nonexistent(cmd, filename)
with open(filename) as f:
    lines=f.readlines()
    for i, line in enumerate(lines):
        print(f'line{i:02}={line}', end='')

However

  • I wanted to try this, because using error handling is the generally considered pythonic way of doing those things and
  • the question is about how to properly combine both of them in order to learn more about python and not about how to work around the example without actually solving the problem.
Camion
  • 1,264
  • 9
  • 22
  • Old question, but did you look into contextlib.suppress? I think it does what you were looking for. Refer to this question as well: https://stackoverflow.com/questions/34566806/why-use-contextlib-suppress-as-opposed-to-try-except-with-pass – Pietro D'Antuono Mar 29 '23 at 08:30

1 Answers1

-2

Your exception handling can be improved, and I would suggest you don't need to use exception handling at all, but just test for non-existence of the file, and can also make use of the file's context manager:

import os
import subprocess
import sys

def myopen(cmd, filename):
    if not os.path.isfile(filename):
        print(f'creating new {filename}', file=sys.stderr, flush=True)
        subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                       text=True, shell=True, executable='/bin/bash')
    
    return open(filename)
Peter Wood
  • 23,859
  • 5
  • 60
  • 99
  • Well, I wanted to use exception handling because it's the generally considered pythonic way to do those things. if I do it by testing file existence first like you do, then I do not even need to rewrite any context manager : `with open(...` is sufficient. – Camion Jul 30 '21 at 11:16
  • @Camion The behaviour of the code is identical. Your use of exception handling added nothing. – Peter Wood Jul 30 '21 at 12:03
  • 1
    The exception handler is better. It avoids a race condition where a file could be created between your attempt to open it and your code to create it. – chepner Jul 30 '21 at 12:37
  • @PeterWood, even if it was the case, this question is not about code behaviour, but about code style and pythonicity. – Camion Jul 30 '21 at 12:48
  • 1
    @chepner No, the behaviour is no different. It only uses exception handling to check whether a file doesn't exist. There is no race condition being handled. – Peter Wood Jul 30 '21 at 16:25
  • @Camion The code is simpler and clearer while retaining the same behaviour. If the term Pythonic means anything, simplicity and clarity are probably part of it. – Peter Wood Jul 30 '21 at 17:01
  • Apologies, I was skipping ahead and thinking about how using `NamedTemporaryFile` would improve this, but that's solving a different issue than what the OP is asking about. – chepner Jul 30 '21 at 17:03
  • But there is nothing un-Pythonic about using exception handling here. – chepner Jul 30 '21 at 17:04
  • @chepner the file not existing is not exceptional, it's expected. There's an idiom of not using exceptions for expected flows. Also, if you handle it as an exception you have to attempt opening it twice. – Peter Wood Jul 30 '21 at 17:08
  • @PeterWood: May I remind you that it is usually requested on stack exchange to make the examples as simple as possible to illustrate the situation. The question is about how to properly combine context managers and exception handling and not about how to defeat the example. – Camion Aug 01 '21 at 09:24
  • @Camion The reason your example is not Pythonically satisfying is because you're making it too complicated. If you insist on keeping it complicated, then it's not Pythonic. The general advice with exceptions is, if you can't handle them, get out of the way and let them whizz past your ears. I'm not missing the point of your example, I'm getting to the essence. Just to note as well, if you've downvoted my answer, and no-one else has answered you, I'm not sure where you're expecting to get help from. I'm not bothered by the points, but I am at least trying to help. – Peter Wood Aug 15 '21 at 11:03
  • @PeterWood, Thank you for you concern, but your working around my example, doesn't teach me how to combine error handling and context management in the same programmatic structure (see title question). What you proposed me is something I was already able to do. If no one else answers, it might probably just mean that my example was already the right way to do what I asked and there is no better way to do it. – Camion Aug 16 '21 at 23:43