What share mode is used when files are opened using open()

Question

I can see nothing in the open() function's parameters that allows specification of how the file will be shared. I suspect therefore that the file will be shared as permissively as possible. Specifically:

When the file is opened for reading, its sharing mode will allow subsequent open operations to open the file for reading, but not for writing.
When the file is opened for writing, its sharing mode will deny subsequent open operations to open the file for reading or writing.

That would seem to me to be the most logical implementation. Are my assumptions correct?

Update: Martijn Pieters states that the answer is OS dependent. So, for the sake of this question, my target OS is Windows.

This depends entirely on the OS; did you have anything in mind? — Martijn Pieters, Dec 02 '13 at 12:31
On Linux, no locking is done, so you can open for reading and writing as many times as you want. I won't post this as an answer since you say you're using Windows, though. — John Zwinck, Dec 02 '13 at 12:34
Related: [How to create a temporary file that can be read by a subprocess?](http://stackoverflow.com/q/15169101/95735) — Piotr Dobrogost, Dec 02 '13 at 18:39

score 8 · Accepted Answer · edited Dec 02 '13 at 13:45

Python uses _wfopen() (Python 2 open() function) or _wopen() (Python 3 and io.open()) internally when opening files, neither of which allow for specifying any sharing flags. Sharing is thus set to a default, and what that default is does not appear to be documented.

If you want to set the sharing mode yourself, you'll have to use the msvcrt.open_osfhandle() if you want to open files specifying a sharing mode.

There is a patch in the Python issue tracker that implements a sharing module, illustrating how to do this. Just the opener from that patch, somewhat simplified, is:

import os
import msvcrt
import _winapi

CREATE_NEW                  = 1
CREATE_ALWAYS               = 2
OPEN_EXISTING               = 3
OPEN_ALWAYS                 = 4
TRUNCATE_EXISTING           = 5
FILE_SHARE_READ             = 0x00000001
FILE_SHARE_WRITE            = 0x00000002
FILE_SHARE_DELETE           = 0x00000004
FILE_SHARE_VALID_FLAGS      = 0x00000007
FILE_ATTRIBUTE_READONLY     = 0x00000001
FILE_ATTRIBUTE_NORMAL       = 0x00000080
FILE_ATTRIBUTE_TEMPORARY    = 0x00000100
FILE_FLAG_DELETE_ON_CLOSE   = 0x04000000
FILE_FLAG_SEQUENTIAL_SCAN   = 0x08000000
FILE_FLAG_RANDOM_ACCESS     = 0x10000000
GENERIC_READ                = 0x80000000
GENERIC_WRITE               = 0x40000000
DELETE                      = 0x00010000
NULL                        = 0

_ACCESS_MASK = os.O_RDONLY | os.O_WRONLY | os.O_RDWR
_ACCESS_MAP  = {os.O_RDONLY : GENERIC_READ,
                os.O_WRONLY : GENERIC_WRITE,
                os.O_RDWR   : GENERIC_READ | GENERIC_WRITE}

_CREATE_MASK = os.O_CREAT | os.O_EXCL | os.O_TRUNC
_CREATE_MAP  = {0                                   : OPEN_EXISTING,
                os.O_EXCL                           : OPEN_EXISTING,
                os.O_CREAT                          : OPEN_ALWAYS,
                os.O_CREAT | os.O_EXCL              : CREATE_NEW,
                os.O_CREAT | os.O_TRUNC | os.O_EXCL : CREATE_NEW,
                os.O_TRUNC                          : TRUNCATE_EXISTING,
                os.O_TRUNC | os.O_EXCL              : TRUNCATE_EXISTING,
                os.O_CREAT | os.O_TRUNC             : CREATE_ALWAYS}


def os_open(file, flags, mode=0o777,
            *, share_flags=FILE_SHARE_VALID_FLAGS):
    '''
    Replacement for os.open() allowing moving or unlinking before closing
    '''
    if not isinstance(flags, int) and mode >= 0:
        raise ValueError('bad flags: %r' % flags)

    if not isinstance(mode, int) and mode >= 0:
        raise ValueError('bad mode: %r' % mode)

    if share_flags & ~FILE_SHARE_VALID_FLAGS:
        raise ValueError('bad share_flags: %r' % share_flags)

    access_flags = _ACCESS_MAP[flags & _ACCESS_MASK]
    create_flags = _CREATE_MAP[flags & _CREATE_MASK]
    attrib_flags = FILE_ATTRIBUTE_NORMAL

    if flags & os.O_CREAT and mode & ~0o444 == 0:
        attrib_flags = FILE_ATTRIBUTE_READONLY

    if flags & os.O_TEMPORARY:
        share_flags |= FILE_SHARE_DELETE
        attrib_flags |= FILE_FLAG_DELETE_ON_CLOSE
        access_flags |= DELETE

    if flags & os.O_SHORT_LIVED:
        attrib_flags |= FILE_ATTRIBUTE_TEMPORARY

    if flags & os.O_SEQUENTIAL:
        attrib_flags |= FILE_FLAG_SEQUENTIAL_SCAN

    if flags & os.O_RANDOM:
        attrib_flags |= FILE_FLAG_RANDOM_ACCESS

    h = _winapi.CreateFile(file, access_flags, share_flags, NULL,
                           create_flags, attrib_flags, NULL)
    return msvcrt.open_osfhandle(h, flags | os.O_NOINHERIT)

OK, but do you know what ultimately gets passed to `CreateFile`? Is it `FILE_SHARE_READ | FILE_SHARE_WRITE`, irrespective of Python file mode? — David Heffernan, Dec 02 '13 at 13:12
@DavidHeffernan: In the default `open()` command on Python 2 you mean? No `FILE_SHARE` flags are set. At all; the `open()` command uses the [`_wfopen()` C API call](http://msdn.microsoft.com/en-us/library/yeby3zcb(v=vs.71).ASPx) instead. — Martijn Pieters, Dec 02 '13 at 13:19
That cannot be quite right. If no `FILE_SHARE` flags are set then the file is opened exclusively. Does this all boil down to the fact that the underlying CRT knows nothing of sharing? And so is my question really a question about `_wfopen()`? — David Heffernan, Dec 02 '13 at 13:21
I suspect so, yes. Incidentally, `io.open()` uses [`_wopen`](http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx), and both documents are silent on what sharing mode is set. — Martijn Pieters, Dec 02 '13 at 13:24
Note that the `_wfopen()` and `_wopen()` methods *probably* set a default sharing mode based on the mode chosen. What that default is does not appear to be documented, however. — Martijn Pieters, Dec 02 '13 at 13:30
This seems like a rum do. Hard to believe Python has no support for locking and sharing. — David Heffernan, Dec 02 '13 at 13:46

score 4 · Answer 2 · answered Dec 02 '13 at 13:07

When the file is opened for writing, its sharing mode will deny subsequent open operations to open the file for reading or writing.

False. No locks are acquired. You can try from two different cmd windows:

first

python -c "import time; f = open('tst','wb'); f.write('1'); time.sleep(3); f.write('333'); f.close"

second, while first is running

python -c "with open('tst','wb') as f: f.write('22')"

results in 1333 in tst file. Flushing writes with f.flush() just after first write in first script results in 2333. Missed data is replaced with \x00 charackters, you can check it for example replacing first f.write('1') with f.write('1'*10**6)

What share mode is used when files are opened using open()

2 Answers2