What's going on with closed file descriptors?

Question

In the example code below we open a file descriptor to sandbox.log, provide it as stdout to a subprocess, and then close the file descriptor, yet the subprocess can still write to the file. Is subprocess.Popen duping the file descriptor internally? Is it safe to close a file descriptor after passing it to a subprocess?

import subprocess
import os
import time


print 'create or clear sandbox.log'
subprocess.call('touch sandbox.log', shell=True)
subprocess.call('echo "" > sandbox.log', shell=True)

print 'open the file descriptor'
fd = os.open('sandbox.log', os.O_WRONLY)

command = 'sleep 10 && echo "hello world"'
print 'run the command'
p = subprocess.Popen(command, stdout=fd, stderr=subprocess.STDOUT, shell=True)

os.close(fd)

try:
    os.close(fd)
except OSError:
    print 'fd is already closed'
else:
    print 'fd takes some time to close'

if p.poll() is None:
    print 'p isnt finished, but fd is closed'

p.wait()
print 'p just finished'

with open('sandbox.log') as f:
    if any('hello world' in line for line in f):
        raise Exception("There's text in sandbox.log. Whats going on?")

For reference, I got the following output running the above code as a script:

% python test_close_fd.py
create or clear sandbox.log
open the file descriptor
run the command
fd is already closed
p isnt finished, but fd is closed
p just finished
Traceback (most recent call last):
  File "test_close_fd.py", line 34, in <module>
    raise Exception("There's text in sandbox.log. Whats going on?")
Exception: There's text in sandbox.log. Whats going on?

You are using Python 2 by the looks of it. See `close_fds` parameter to `Popen`. The default changed in 3.2 for these reasons. — cdarke, Jun 08 '16 at 22:02

score 6 · Accepted Answer · answered Jun 08 '16 at 21:56

Each process has its own set of file descriptors. Closing an fd in one program will not affect another program. This is why every program can use the same fd numbers for stdin (0), stdout (1), and stderr (2), and why shell scripts can usually just open fd 3 without having to check if it's available.

File descriptors are inherited by child processes, unless you explicitly prevent it by setting the close-on-exec flag. By default, without that flag, a child process will get copies of the parent's file descriptors.

Thanks John. Your answer was very enlightening. – chmod 777 j Jun 08 '16 at 21:59 — chmod 777 j, Jun 08 '16 at 21:59

What's going on with closed file descriptors?

1 Answers1