1 in every 10000 times or so I get a PermissionError when using os.remove()

Question

I wrote a script where I'm extracting frames from a video to get some info, so I loop over the frames, write the image to a temporary folder, do some operations, delete the image:

# if frame exists, capture 
while framesleft:
    framesleft,image = vid.read()
    if framesleft:
        imfile = cv2.imwrite("temperonispaghettioni/video%03dframe%05d.png" % (videonr, framenr), image)

        # write the filename, video and frame to FrameList.csv
        row = ["bla bla bla unimportant stuff"]
        writer.writerow(row)  

        # Make sure the image is closed and remove the image     
        image = None
        imfile = None
        os.remove("temperonispaghettioni/video%03dframe%05d.png" % (videonr, framenr))

Most of the times this goes perfectly fine. Once in every 10000 iterations or so, not an exact number and it varies, I get the following errror:

    os.remove("temperonispaghettioni/video%03dframe%05d.png" % (videonr, framenr))

PermissionError: [WinError 5] Access is denied: 'temperonispaghettioni/video000frame01218.png'

As you can see I tried setting image and imfile to None in an attempt to close the file. This doesn't seem to solve the issue. What else could I try to correctly close the file, or is there something else wrong here?

Also my while framesleft: if framesleft: is an inefficient way of doing this, I realise that. The problem I had is that when the video was out of frames the code would create 1 last corrupted .png that I can't process. If anyone has a suggestion for improvements this would be very welcome.

EDIT: I've try-catched the original error:

try:
    os.remove("temperonispaghettioni/video%03dframe%05d.png" % (videonr, framenr))
except PermissionError:
    print('Ran into an error with deleting frame %05d of video %03d, trying again...' %(framenr,videonr))
    time.sleep(0.5)
    try:
        os.remove("temperonispaghettioni/video%03dframe%05d.png" % (videonr, framenr))
        print('Succesfully removed frame%05d of video %03d, moving on...' %(framenr,videonr))
    except PermissionError:
        print('Removal failed, frame will remain in the folder and the folder won\'t delete itself. Moving on...')

I've looped the entire script to run a couple hundred times to see if I can recreate the error, so far it doesn't even go to the except stage, but I got a new error:

 os.mkdir('temperonispaghettioni')

PermissionError: [WinError 5] Access is denied: 'temperonispaghettioni'

Same issue, but now for making the directory. Should I just try-catch every writing/deleting operation?

Note that `cv2.imwrite` returns a boolean that tells you whether the instruction was successful or not. Setting that return value to `None` does not do anything, and especially not close the image. It seems that `imwrite` will already close the underlying file handle automatically. — poke, Nov 27 '19 at 14:11
Do you have some sort of background indexer or virus scanner that might take an interest in your new files? Windows in general does not support deleting files that are “in use”. — Davis Herring, Nov 27 '19 at 14:26

score 2 · Accepted Answer · answered Nov 27 '19 at 15:05

Your construction here requires a middle loop break:

while True:
    framesleft, image = vid.read()
    if not framesleft:
        break
    process(framesleft, image)

Another way, if you don't mind the little bit of repetition, you can also do the first read outside the loop:

framesleft, image = vid.read()
while framesleft:
    process(framesleft, image)
    framesleft, image = vid.read()

Before Python 3.8, those are really the only way to write that kind of loop in Python.

In Python 3.8 and above though, Python gained a new assignment expression syntax that people have been calling the walrus operator: :=. With the walrus operator you can instead do something like this:

while (frame_image := vid.read())[0]:
    framesleft, image = frame_image
    process(framesleft, image)

As to why you're getting permission denied error, well I can't really say, there isn't enough information in your post to really know what's happening here. Did you make sure that the directory was empty before you started the program, and that there are no leftover files from previous runs that might have incorrect permission? Do you have another instance of the program running in the background or in a separate thread/process?

Also, since you're creating the file only to delete it afterwards, why not just use https://docs.python.org/3.7/library/tempfile.html#tempfile.mkstemp so that you don't have to generate unique filenames and pray that there's no collisions or permissions issues.

Thanks, this cleaned up my code a bit. Is there a speed benefit to using temp files vs writing and deleting them? — fristiprinses, Nov 27 '19 at 15:47
@fristiprinses: I wouldn't expect any speed benefits, but why don't you just try them and measure? The main reason I recommended using temp files is not speed but rather to avoid some common pitfalls when creating temporary files, such as race conditions. — Lie Ryan, Nov 27 '19 at 16:46

1 in every 10000 times or so I get a PermissionError when using os.remove()

1 Answers1