Detecting broken stream in python when file is deleted

Question

My problem is that logging stops for a python program when the log is rotated. I have tracked it down to the stream itself. I don't see any way to tell if the stream is broken from python. After the file is deleted it still accepts writes without any issue.

import os

FILE = 'testing.txt'

fs = open(FILE, 'a')
fs.write('word')
os.remove(FILE)
fs.write('Nothing....') # Nothing breaks
print(fs.errors) # No errors

So, how can I find out if the file stream is still valid? And checking to see if the file exists will not help since the file will always exist regardless of whether or not the stream is still valid.

What os and what version of python are you using? I tried running this and os.remove(FILE) gave me an error; it said the file was in use! I didn't have it open in any other program, so it must have been it being open in python. I'm using XP on this laptop and as it is my school computer it only has Python 2.5 — Curlystraw, May 10 '11 at 19:37
Interesting. My target os is Linux (redhat) and Python 2.6.5. And I tested the code in Linux (Ubuntu) Python 2.7.1 — Bryan Austin, May 10 '11 at 19:45

score 2 · Accepted Answer · answered May 11 '11 at 13:41

Upon much more inspection, I found the solution. It is an OS specific problem. When the file is deleted in Linux (or Macintosh) it just unlinks it. (I was not aware of this) So if you run lsof on the machine, it still shows the file as open.

[user@machine]$ lsof | grep --color -i "testing.txt"
python26  26495    user    8w      REG               8,33     23474     671920 /home/user/temp/testing.txt (deleted)

The solution is to stat the stream in python.

stat = os.fstat(fs.fileno())

Which will give you the number of links it has.

if stat.st_nlink < 1:
    #has been deleted

And there you go. Now you know if you should reload it or not. Hopefully this helps someone else.

score 0 · Answer 2 · answered May 10 '11 at 19:18

0

Try Exception handling:

import os

FILE = 'testing.txt'

try:
    fs = open(FILE, 'a')
    fs.write('word')
    os.remove(FILE)
    fs.write('Nothing....') # Nothing breaks
except Exception, e:
    print "Error:", e
print(fs.errors) # No errors

answered May 10 '11 at 19:18

abaumg

2,083
1
16
24

score 0 · Answer 3 · answered May 10 '11 at 19:28

0

There are python bindings for ionotify if you need more intelligence than just an try: except: clause. But I think its only pertinent to Linux (im not sure of your platform)

answered May 10 '11 at 19:28

tMC

18,105
14
62
98

score 0 · Answer 4 · answered Aug 15 '11 at 14:01

0

Another solution I found is to add the "copytruncate" flag into the logrotate config. See "man logrotate" for more info.

answered Aug 15 '11 at 14:01

Bryan Austin

487
3
13

Detecting broken stream in python when file is deleted

4 Answers4