4

I'm working on some Python code which is a CGI script called from Apache.

The first thing the code does is (I believe) attempt to close stdin/stdout/stderr with the following:

    for fd in [0, 1, 2]:
    try:
        os.close(fd)
    except Exception:
        pass

Normally this works, however if they're not open, I get a "python.exe has stopped working", "A problem caused the program to stop working correctly" error message (Win32 exception).

Couple of questions:

  • What's the difference between closing via os.close(descriptor number) and sys.stdin.close() etc.
  • Assuming I should be closing via both mechanisms, how can I check that the descriptor is actually open (i.e. invoking os.close won't cause Python to crash)
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
eddiewould
  • 1,555
  • 16
  • 36

1 Answers1

8

I don't know for sure, but I would bet that os.close() is just a python wrapper for the OS's close() system call. Python file objects handle some cool stuff for the user like internal buffering of data and whatnot and this is taken care of when calling its close() method.

Eventually, the OS's close() system call will be called on the File object's file descriptor. So, calling sys.stdin.close() at some point is equivalent to calling os.close(sys.stdin.fileno())

According to the docs, you can call the close() method on a file multiple times and Python won't care. File objects even provide an attribute, closed to check if a file is open or not:

>>> import os

>>> f = open(os.devnull)
>>> f.close()
>>> f.close()
>>> f.close()
>>> f.close()
>>> f.close()
>>> print f.closed
True

If possible, I would suggest calling the sys.FD's close() method as it's cleaner and more pythonic.

Update

After looking at the source for python, I found this for file objects (fileobjects.c):

static PyObject *
file_close(PyFileObject *f)
{
    PyObject *sts = close_the_file(f);
    if (sts) {
        PyMem_Free(f->f_setbuf);
        f->f_setbuf = NULL;
    }
    return sts;
}

PyDoc_STRVAR(close_doc,
"close() -> None or (perhaps) an integer.  Close the file.\n"
"\n"
"Sets data attribute .closed to True.  A closed file cannot be used for\n"
"further I/O operations.  close() may be called more than once without\n"
"error.  Some kinds of file objects (for example, opened by popen())\n"
"may return an exit status upon closing.");

Inside close_the_file(f);

static PyObject *
close_the_file(PyFileObject *f)
{
    int sts = 0;
    int (*local_close)(FILE *);
    FILE *local_fp = f->f_fp;
    char *local_setbuf = f->f_setbuf;
    if (local_fp != NULL) {
        local_close = f->f_close; // get fs close() method

        /* SOME CONCURRENCY STUFF HERE... */

        f->f_fp = NULL;
        if (local_close != NULL) {
            f->f_setbuf = NULL;
            Py_BEGIN_ALLOW_THREADS
            errno = 0;
            sts = (*local_close)(local_fp); // Call the close()
            Py_END_ALLOW_THREADS
            f->f_setbuf = local_setbuf;
            if (sts == EOF)
                return PyErr_SetFromErrno(PyExc_IOError);
            if (sts != 0)
                return PyInt_FromLong((long)sts);
        }
    }
    Py_RETURN_NONE;
}

What is the file's close() method?

static PyObject *
fill_file_fields(PyFileObject *f, FILE *fp, PyObject *name, char *mode,
                 int (*close)(FILE *))
{
    ...
    f->f_close = close;
    ...
}

And for the os module (which uses posixmodule.c):

/* This file is also used for Windows NT/MS-Win and OS/2.  In that case the
   module actually calls itself 'nt' or 'os2', not 'posix', and a few
   functions are either unimplemented or implemented differently.  The source
   assumes that for Windows NT, the macro 'MS_WINDOWS' is defined independent
   of the compiler used.  Different compilers define their own feature
   test macro, e.g. '__BORLANDC__' or '_MSC_VER'.  For OS/2, the compiler
   independent macro PYOS_OS2 should be defined.  On OS/2 the default
   compiler is assumed to be IBMs VisualAge C++ (VACPP).  PYCC_GCC is used
   as the compiler specific macro for the EMX port of gcc to OS/2. */

PyDoc_STRVAR(posix_close__doc__,
"close(fd)\n\n\
Close a file descriptor (for low level IO).");

/*
The underscore at end of function name avoids a name clash with the libc
function posix_close.
*/
static PyObject *
posix_close_(PyObject *self, PyObject *args)
{
    int fd, res;
    if (!PyArg_ParseTuple(args, "i:close", &fd))
        return NULL;
    if (!_PyVerify_fd(fd))
        return posix_error();
    Py_BEGIN_ALLOW_THREADS
    res = close(fd);  // close the file descriptor fd
    Py_END_ALLOW_THREADS
    if (res < 0)
        return posix_error(); // AKA raise OSError()
    Py_INCREF(Py_None);
    return Py_None;
}

If the file descriptor has already been closed, then an OSError is raised:

static PyObject *
posix_error(void)
{
    return PyErr_SetFromErrno(PyExc_OSError);
}

So, calling os.close(fd) should raise an OSError if called twice on the same file descriptor. Calling file.close() eventually calls fclose(FILE *f) and will handle it being called multiple times.

OozeMeister
  • 4,638
  • 1
  • 23
  • 31
  • The developer who wrote the code I'm working on really knew what he was doing (however he's left the organisation so I can't ask him). I suspect he had a reason for doing it the way he did. – eddiewould Jun 11 '14 at 23:57
  • 1
    The only other thing I can think of is that perhaps they switched the FDs of std in, out, and err using `os.dup` and `os.dup2` and are closing the files that point to those FDs. You could try using `os.fdopen()` on the FDs then to get a File object and then call its `close()` method, but that's kinda dumb since calling `os.close()` should work. If neither `os.dup` or `os.dup2` have been called, then std in, out, and err should still point to FDs 0, 1, and 2 respectively. – OozeMeister Jun 12 '14 at 00:44
  • I'll leave the question open for a while to see if anyone else chimes in, otherwise will accept your answer. – eddiewould Jun 12 '14 at 00:49