85

I am using a scientific software including a Python script that is calling os.system() which is used to run another scientific program. While the subprocess is running, Python at some point prints the following:

close failed in file object destructor:
IOError: [Errno 9] Bad file descriptor

I believe that this message is printed at the same time as os.system() returns.

My questions now are:

Which conditions can lead to this type of IOError? What does it exactly mean? What does it mean for the subprocess that has been invoked by os.system()?

Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
Dr. Jan-Philip Gehrcke
  • 33,287
  • 14
  • 85
  • 130

3 Answers3

69

You get this error message if a Python file was closed from "the outside", i.e. not from the file object's close() method:

>>> f = open(".bashrc")
>>> os.close(f.fileno())
>>> del f
close failed in file object destructor:
IOError: [Errno 9] Bad file descriptor

The line del f deletes the last reference to the file object, causing its destructor file.__del__ to be called. The internal state of the file object indicates the file is still open since f.close() was never called, so the destructor tries to close the file. The OS subsequently throws an error because of the attempt to close a file that's not open.

Since the implementation of os.system() does not create any Python file objects, it does not seem likely that the system() call is the origin of the error. Maybe you could show a bit more code?

Sven Marnach
  • 574,206
  • 118
  • 941
  • 841
  • 1
    Thanks.. I already had this suspicion. Showing more code would not make sense at this point, because this Python script is quite big, not well-structured and not well-written. I don't have a very good overview so far. With the help of your information, I'll try to find the issue myself first. – Dr. Jan-Philip Gehrcke Oct 07 '11 at 11:18
  • One more question: in your example, you used `del f` instead of `f.close()`, which would have resulted in a traceback. Is `del f` equivalent with what happens during Python's garbage collecting? – Dr. Jan-Philip Gehrcke Oct 07 '11 at 11:21
  • My first guess is that it has something to do with the script making use of MPI parallelization -- i.e. shared file descriptors between different processes. – Dr. Jan-Philip Gehrcke Oct 07 '11 at 11:26
  • 1
    @Jan-PhilipGehrcke: `del f` simply deletes the name `f`, decreasing the reference count of the object that name points to. If there are no other references left, this triggers garbage collection, including a call to the `__del__()` method of the object. Since you were getting the error message in a `os.system()` call, I suspect it is due to implicit garbage collection, so I used `del` to demonstrate the effect. The single line `os.close(open(".bashrc").fileno())` produces the same error. – Sven Marnach Oct 07 '11 at 11:26
  • 2
    @Jan-PhilipGehrcke: If you are using Python 2.6 or 2.7, you could try a hack: `import io; __builtins__.open = io.open` replaces the `open()` builtin by the version from the `io` module (backported from 3.0). The new version of `open()` is slower, but ignores if the file is already closed. – Sven Marnach Oct 07 '11 at 11:29
  • @Sven, in your single line example the order is reversed – first the object is destroyed, then the descriptor is closed. It doesn't give the “close failed in file object destructor” message, but a regular exception. – Petr Viktorin Oct 07 '11 at 11:32
  • @PetrViktorin: You are right -- thanks for pointing this out. – Sven Marnach Oct 07 '11 at 11:40
  • @Sven and @Petr: Sven said that this error occurs if the file was not closed by its file object's `close()` method. Could it be that two `close()` calls from different MPI processes also lead to the error (without traceback)? – Dr. Jan-Philip Gehrcke Oct 07 '11 at 11:42
  • 1
    @Jan-Philip: Yes, if the MPI passes around OS-level file descriptors and not Python file objects. If you're using a MPI, I'd call this a very probable cause. – Petr Viktorin Oct 07 '11 at 11:56
  • I was using a neo4j database and got this problem. Turns out i wasn't closing the connection after sending my requests. Calling the "close()" function on the db driver object did the trick. Thanks. – Loïc N. Nov 02 '21 at 08:34
  • I think my issue was that I wasn't passing an additional parameter into a function when the file was still open. So the other error somehow caused this one. – Dave Liu Dec 08 '21 at 06:26
19

You can get this error if you use wrong mode when opening the file. For example:

with open(output, 'wb') as output_file:
    print(output_file.read())

In that code, I want to read the file, but I use mode wb instead of r or r+

vvvvv
  • 25,404
  • 19
  • 49
  • 81
Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
-1

You can get this error if your Ransomware Protection is enabled on your working directory. Windows does not allow any third party applications to make changes to files if the directory is protected by Ransomware Protection which comes with windows. You can rectify this by going to "Windows Security -> Virus and Threat Protection -> Manage Ransomware Protection -> Allow an app through controlled folder access"

Then add "Python[version].exe" by clicking Add an allowed app.

Wai Ha Lee
  • 8,598
  • 83
  • 57
  • 92
Jerin BS
  • 41
  • 1
  • 2
    and that's how someone can screw you up with a python ransomeware script ladies and gentlemen! – John C. Aug 01 '22 at 18:47
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Aug 02 '22 at 21:58
  • 2
    I know that people are downvoting this for being insecure, but in my specific case, this was actually what was causing the problem. I simply turn controlled folder access off before running my script, and then back on again immediately after it finishes. – Darkstar Feb 28 '23 at 16:21
  • In my case, I was running a compiled Python application that modifies ini files. For whatever reason, on one computer, a Windows security setting was preventing file modification of those ini files (my compiled application runs on ~100 computers, and I have only encountered this issue on one - likely related to some other security setting set by the computer's owner). Adding a security exclusion to the path to my compiled application got around the issue. – Brian K Mar 01 '23 at 19:58
  • This helped me as well. – Outsider Mar 13 '23 at 16:08