0

I'm having trouble monitoring a file for changes. I need to be able to know when a file changes, and when it does, I need the new line that was added. I intend to parse each line and find ones that match certain criteria, and act on information in those lines. I know the expected number of matching lines ahead of time, but I do not know how many lines in total will be added to the file, or where the matching lines will be.

I've tried 2 packages so far, with no avail.

fsnotify/fsnotify

As fas as I can tell, fsnotify can only tell me when a file is modified, not what the details of the modification was. Since I need to know what exactly was added to the file, this is no good for me.

(As a side-question, can this be run in a loop? The example that I tried exited after just one modification. I need to monitor for multiple modifications.)

hpcloud/tail

This package tries to mimic the Unix tail command, but it seems to have its own issues. The output that I get includes timestamps and other data - I just want the added line, nothing else. Also, it seems to think a file has been modified multiple times, even when it's just one edit. Further, the deal breaker here is that it does not output the last line if the line was not followed by a newline character.

Delegating to tail

I came across this answer, which suggests to delegate this work to the tail command itself, but I need this to work cross-platform (specifically, macOS, Linux and Windows). I don't believe that an equivalent command exists on Windows.

How do I go about tackling this?

Community
  • 1
  • 1
tverghis
  • 957
  • 8
  • 31
  • 1
    What wrong with the extra information from `hpcloud/tail`? You still get the appended data. – JimB Dec 01 '16 at 22:51
  • @JimB it's not ideal, but I can work around it. The main reason I cannot use `hpcloud/tail` is because it does not output the last line if it is not followed by a newline. – tverghis Dec 01 '16 at 22:54
  • You can certainly use fsnotify then, you just read the file as needed depending on the notifications received. I don't know of a better complete "tail -f" package written in Go (but offsite recommendations are off topic anyway) – JimB Dec 01 '16 at 22:58
  • See https://stackoverflow.com/questions/31120987/go-tail-f-like-generator#answer-31122253 for a simple solution. It works well when the app is monitoring a small number of files and can tolerate a little latency. – Charlie Tumahai Dec 02 '16 at 01:37
  • @JimB thanks for pointing me in the right direction. I ended up using `fsnotify` along with [go-diff](https://github.com/sergi/go-diff/) to get just the last line whenever `fsnotify` identified a change. If you'd like to write your comment as an answer, I can accept it. – tverghis Dec 02 '16 at 01:41

1 Answers1

0

@user2515526,

Usually changed diff is out of scope of file watchers' functionality, because, you know, you could change an image, and a watcher would need to keep a track several Mb of a diff in memory, and what if we have thousands of files? However, as bad as it sounds, this may be exactly the way you want to implement this (sure, depends on your app, etc. - could be fine for text files), i.e. - keeping a map of diffs (1 diff per file) since last modification. Cannot say I like it, but sounds like fsnotify has no support for changes/diffs that you need.

Also, regarding your question about running in a loop, maybe you can get some hints here: https://github.com/kataras/iris/blob/8370d76910cdd8de043753ed81ae080eae8dc798/utils/file.go Its a framework that allows to build a server that watches for TypeScript file changes. So sounds similar to your case/question.

Cheers, -D

oharlem
  • 1,030
  • 7
  • 12