Open file for read/write, create if needed

Question

What is the most elegant way to open a file such that

the file gets created if it does not exist,
the file won't be truncated if it does exist and
it is possible to write any part of the file (after seeking), not just the end?

As far as I can tell, the open builtin doesn't seem up to the task: it provides various modes, but every one I tried fails to satisfy at least one of my requirements:

r+ fails if the file does not exist.
w+ will truncate the file, losing any existing content.
a+ will force all writes to go to the end of the file, at least on my OS X.

Checking for the existence prior to opening the file feels bad since it leaves room for race conditions. The same holds for retrying the open with a different mode from within an exception handler. I hope there is a better way.

If your goal is to write to the middle of the file, I doubt there is a very clean way. — msvalkon, Mar 03 '14 at 08:54

score 9 · Accepted Answer · answered Mar 03 '14 at 08:56

9

You need to use os.open() to open it at a lower level in the OS than open() allows. In particular, passing os.RDWR | os.O_CREAT as flags should do what you want. See the open(2) man page for details. You can then pass the returned FD to os.fdopen() to get a file object from it.

answered Mar 03 '14 at 08:56

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

1

Particularly in combination with `os.fdopen` this looks very good. I just found abot [this answer](http://stackoverflow.com/a/10352231/1468366) in the related links, which suggests the same. Thanks! – MvG Mar 03 '14 at 08:58
1

I tried `fd = os.open('.pree_config', os.O_RDWR | os.O_CREAT) config_file = os.fdopen(fd)` but I got `io.UnsupportedOperation: not writable` error. I fixed this by passing the 'r+' argument to `os.fdopen()`... – deed02392 Jul 17 '19 at 11:52

falsetru · Answer 2 · 2014-03-03T09:02:47.087

2

If you are using Python 3.3+, you can use x mode (exclusive creation mode):

try:
    f = open('/path/to/file', 'x+')
except FileExistsError:
    f = open('/path/to/file', 'r+')

It raises FileExistsError if the file already exists.

edited Mar 03 '14 at 09:02

answered Mar 03 '14 at 08:56

falsetru

357,413
63
732
636

score 0 · Answer 3 · edited Mar 03 '14 at 08:58

0

I might be wrong, but I don't suppose there is going to be a race condition if there are not multiple threads, and the try and except blocks are the same thread? (Is it actually possible to do multiple threads?)

This should be up to the task.

>>>try: 
      f=open('myfile.txt','r')
   except OSError:
      f=open('myfile.txt','w')
   finally:
      #whatever file I/O you need.

edited Mar 03 '14 at 08:58

Balthazar Rouberol

6,822
2
35
41

answered Mar 03 '14 at 08:55

Guy

604
5
21

I'm concerned about multiple applications, some perhaps using different code. – MvG Mar 03 '14 at 08:57
"The same holds for retrying the open with a different mode from within an exception handler." And the race condition can happen with other programs regardless of the use of threading in the Python script. – Ignacio Vazquez-Abrams Mar 03 '14 at 08:57
@IgnacioVazquez-Abrams I don't follow, other programs are trying to access the file? Then shouldn't the checking be done in *that* program and not the Python script? I might be missing something embarrassingly obvious. – Guy Mar 03 '14 at 08:58
@IgnacioVazquez-Abrams Anyway your answer seems to have covered it. +1 – Guy Mar 03 '14 at 08:59
1

An exploit can attempt to take advantage of the small time between `open()` calls to e.g. link in a system file, destroying it in the process. – Ignacio Vazquez-Abrams Mar 03 '14 at 09:00
@IgnacioVazquez-Abrams Oh. I do not usually tend to think about intentionally malicious users, I'm naive that way I guess. Also check falsetru's solution. Looks good. – Guy Mar 03 '14 at 09:01

score 0 · Answer 4 · answered Mar 03 '14 at 13:53

I was having a similar problem when trying to dump items to a file as a dictionary. However, I imported json, http://docs.python.org/2/library/json.html check this out, maybe very helpful. Remember to import json. This will provide the foundation to dump and load data whenever you need to. In this case I am dumping and loading information into an empty dictionary. The try and except method is very useful when you want to use an empty dictionary. I find "r+" most useful since it will read and write the file.

def dump_data():
    j = json.dumps(file.text, indent=4)
    with open("database.txt", "w") as f:
        f.write(j)

def load_data():
    try:
        with open("file.txt", "r+") as f:
            return json.load(fp=f)
    except IOError:
            return {}

Right now I'm writing items in response to interactive activity, and flushing the modifications to disk after every such write, so nothing will get lost in case of a serious crash or power outage. Writing the whole data anew after every modification feels rather inefficient in this setup. — MvG, Mar 03 '14 at 14:46

Open file for read/write, create if needed

4 Answers4

Linked