5

Short description

I want to detect local changes in my local git repository in a performant way. This means detecting which files are modified locally, but also the unversioned ones. Any uncommited change / file addition has to be picked up.

My attempt

What I've gotten so far (gist): https://gist.github.com/kevin-smets/02bda148a5330f350e6b

This is a script which can be run indepentently or as a git plugin. What it does is basically the following, launch a git status --porcelain every two seconds, interpret this to check which files are changed. However, when it's running, it takes about 8% CPU constantly. I tried git --diff-files and all kinds of other commands, but all resulted in a high CPU usage. :(

Why?

I would like to feed this list of files to a grunt watch process. Currently my codebase is around 15k files and growing. Watching all these files will simply become impossible. I already apply a static filter to limit the amount of files watched, but it is getting cumbersome to change this by hand every time I want to edit a file outside of the defined filter.

A thought

A thought I had was, how does IntelliJ / WebStorm (my current IDE) monitor the GIT repository? I tried going into the source code of the community edition but well, I can't figure that one out :). There must be some way WebStorm is doing this without taxing the CPU, it seems to pick up these changes on a timeout of a second or less... Even if they were made outside of the IDE. I was wondering if there is some eventing system in GIT, but I can't find anything along those lines.

Thanks for reading

Any help, is always highly appreciated!

smets.kevin
  • 1,570
  • 14
  • 17
  • 2
    Search about filesystem hooks. That allows you to set actions (your git command in this case) to be executed whenever a change occurs in the filesystem – Manu343726 Jul 09 '14 at 14:22
  • Have you thought of using something like https://github.com/rvoicilas/inotify-tools/wiki to monitor files for changes, then every time you detect a change, you simply check with `git` to see if the file is being tracked? – Attila O. Jul 09 '14 at 14:22

2 Answers2

1

I'm using:

git status -s --porcelain

Benchmark

time git status -s --porcelain
#...
real    0m0.078s
user    0m0.032s
sys 0m0.011s

Simple:

time git status

#...
real    0m0.182s
user    0m0.075s
sys 0m0.089s
gpupo
  • 942
  • 9
  • 16
  • Hmmm, that does improve it somewhat. From 8% to 5,4%. I guess I will need to look into filesystem hooks. I wanted to make a universal solution (non OS specific), so relying only on GIT. However this does not seem to be a possibility. Thanks for the suggestion nonetheless! – smets.kevin Jul 09 '14 at 17:57
0

I eventually went with Chokidar to monitor my source folder. It's now at around 20k files and as long as I watch the folder itself and no globs like **/*.scss, it works like a charm.

I do the filtering on filetype myself by checking the extname of the files and feeding it to the correct build target. This way I eliminated git from the process. You could still run any git command when Chokidar notifies you of a file change, this way the ugly arbitrary timeout is eliminated and your CPU will not get taxed constantly.

smets.kevin
  • 1,570
  • 14
  • 17