How to follow a large file in python without loading it all in the memory?

Question

I tried the solution in this answer, since the file I'm tailing can grow large than 50GB, the server is choking. Any suggestions on how to follow it without storing all in memory?

Answer 1

You don't need an external app.

import time
fsl = open('/var/syslog')
# Seek to end
fsl.seek(0,2)
last = fsl.tell()
# Back up 200 bytes
fsl.seek( last-200 )
while True:
    print( fsl.read() )
    time.sleep(5)

Now, this will print an extra linefeed every time it tries to get more data, but you can fix that.