4

I've seen a few questions related to this but nothing that definitively answers my question.

I have a short python script that does some simple tasks, then outputs some text to a log file, waits for more input, and loops.

At times, the file is opened in write mode ("w") and other times it is opened in append mode ("a") depending on the results of the other tasks. For the sake of simplicity let's say it is in write mode/append mode 50/50.

I am opening files by saying:

with open(fileName, mode) as file:

and writing to them by saying:

file.write(line)

While these files are being opened, written to, appended to, etc., I expect a command prompt to be doing some read activities on them (findstr, specifically).

1) What's going to happen if my script tries to write to the same file the command window is reading from?

2) Is there a way to explicitly set the open to shared mode?

3)Does using the 'logger' module help at all/handle this instead of just manually making my own log files?

Thanks

Mike T
  • 482
  • 1
  • 6
  • 16

1 Answers1

3

What you are referring to is generally called a "race condition" where two programs are trying to read / write the same file at the same time. Some operating systems can help you avoid this by implementing a file-lock mutex system, but on most operating systems you just get a corrupted file, a crashed program, or both.

Here's an interesting article talking about how to avoid race conditions in python: http://blog.gocept.com/2013/07/15/reliable-file-updates-with-python/

One suggestion that the author makes is to copy the file to a temp file, make your writes/appends there and then move the file back. Race conditions happen when files are kept open for a long time, this way you are never actually opening the main file in python, so the only point at which a collision could occur is during the OS copy / move operations, which are much faster.

Mike Ounsworth
  • 2,444
  • 21
  • 28
  • 1
    After a little more investigating and experimentation, I can cautiously confirm that python will allow a command window to read from a file that is currently open for write/append. Can't speak for other processes – Mike T Nov 17 '14 at 21:49
  • 1
    Cool! If one process is only doing reads then that's probably safe anyways. Problems happen when two or more things are trying to write at the same time. – Mike Ounsworth Nov 17 '14 at 21:50
  • 2
    3000 views and 5 years of experience later, I hope people who are coming here are trying your temp file approach, which is definitely safer than what I commented above – Mike T Oct 30 '19 at 18:34