node.js script running in VM doesn't see folder changes made from main OS

Question

I have following setup:

1. Ubuntu Lucid 32 VM created by Vagrant.
2. Folder from which Vagrant is started with some CSS and JS files. The folder is mounted inside VM at address /vagrant/.
3. grunt.js with watch task started from VM, watching for changes in css and js files. After the files are changed, it must concat this files to one css and one js file.

The problem: Watch task in grunt.js didn't see changes in mounted folder.

If I change files inside VM, watch task work normally. If I start grunt.js not from virtual box, but from main os it also works. Tried in Mac OS and Windows 7.

Problem only happens then task is started from VM, and files are changed from main OS. Is there a workaround to this problem?

Update. Some details about our usage scenarios, to better understand context:

1. We are desing studio, with 10+ peaple working with the code.
2. Different people using different OS. We have: Win 7, Mc OS X and Ubuntu Linux.
3. We are working on and supporting 100+ different projects.
4. Prjects are using different languages and setups: We have PHP projects on our own CMS, PHP projects on Yii, Python projects on Django, and some node.js projects.
5. Different developers may at some point of time work on any of this projects.
6. Right now we spend a lot of time every time we need to make project work on computer of the developer, who didn't work with it before. And usually we need some help of developer who already worked on it to make it work.

We want to make it more straightworward. For it we trying to use Vagrant and a number of pre-configured OS images.

Right now project setup looks like this:

1. Developer checkout project from git.
2. Developer opens project folder in Terminal.
3. Developer enters 'vagrant up'.
4. After 5 minutes he have fully configured and working project copy.

We like this setup and most of the time we didn't have any problems with it. But on our node.js projects we use some css and js preprocessors, and like to update their files in realtime. And there is the problem on initial post.

Solutions with access to files with ssh VM and with project's files inside VM will probably work, but they didn't solve the base problem (fast and easy setup of big number of projects across group of developer running different OS'es)

Answer 1

File/Path watchers depend on OS's lower level facilities like inotify on Linux or kqueue on BSD systems. These mechanisms work directly with the filesystem drivers and they will not see the changes on network mounts. If your script will work on the virtual OS, then you might change the direction of sharing. Instead of reaching a shared folder on host OS from inside the guest OS, use a real folder on guest OS (where the real action happens) and reach it from the host OS to edit your files. This way your guest OS will work as it would on its production environment, which is what Vagrant promises; but fails to deliver, apparently.

I personally use Sublime Text 2 editor with SFTP plugin. In this setup, I keep two copies of files and work on the ones on the host OS. Whenever I edit/add a file, my editor quickly uploads it to server, i.e the guest OS. When I move to production servers, I can still use the same approach which is better than using shared folders.

Before switching to Sublime as my main editor, I had been using Vim for more than a decade. At that time, instead of using Vim's sftp/ssh facilities, which were always buggy, I was using WinSCP as SFTP client. After you configure it properly, whenever you double click a file on the server, it opens it on your choice of editor and whenever you save the file, WinSCP automatically uploads it to server. This was very close to what I have with Sublime's SFTP plugin; however the local (host OS) versions of files were created on temporary folders WinSCP creates on the fly.

The approach Sublime's SFTP plugin has, which is having duplicates of folders on both systems fits my needs better because I have also SpiderOak running as my backup solution on the host OS. I love it since it keeps version history of files. I set it up to check changes once an hour, instead of constantly watching to make it perform smoother risking losing work of an hour in case of a system failure. Everything works perfectly in this setup and I have really used its old-versions feature more than a few times which saved me from a lot of drudgery and loss of work.

So, in summary, if you want to continue working on shared folders, what you will do is to install and configure a Samba/NFS/FTP server on guest OS (your Ubuntu) and share the folders. Then you will access the network share from host OS (your Mac/Win7) the same way you access any other shared remote folders. If your use and needs are similar to mine, then I strongly recommend you going with either Sublime+SFTP or YourEditor and WinSCP or a similar SFTP client.

UPDATE: Based on especially the symptom summarized as "If I change files inside VM, watch task work normally. If I start grunt.js not from virtual box, but from main os it also works." in the question, I made a wrong assumption about how grunt.js watches files and folders. I thought it was using node's fs.*watch functions; but as pyfunc showed with a well done research, my assumption was wrong. Although this does not affect the outcome of any suggestion I made, it was a wrong technical information I have to apologize for. So, it looks like the problem is caused by the modification times on files not being set correctly when modification is done on remote machine which may render vboxfs useless for this kind of use that depends on modification times of files.

Answer 2

Note: I am not presenting any complete solution and I am no JavaScript programmer

Section on grunt.js issue on watching file

Source: https://github.com/cowboy/grunt/blob/master/tasks/watch.js

Section where it performs watching of file. It uses fs.statSync (Synchronous stat):

// Get last modified time of file.
            mtime = +fs.statSync(filepath).mtime;
            // If same as stored mtime, the file hasn't changed.
            if (mtime === mtimes[filepath]) { return; }
            // Otherwise it has, store mtime for later use.
            mtimes[filepath] = mtime;
          }

It uses "fs" functionality from node.js: http://nodejs.org/api/fs.html#fs_class_fs_stats

Among various data on file, it returns following time attributes:

atime: Mon, 10 Oct 2011 23:24:11 GMT,
mtime: Mon, 10 Oct 2011 23:24:11 GMT,
ctime: Mon, 10 Oct 2011 23:24:11 GMT 

So conclusion is that it uses stat. You can write a simple js file leveraging fs from node.js to check the fs.statSync result before and after changing the file in a shared folder.

This would establish if you should be watching file with watcher utility from grunt.js.

A little note on shared folders in VirtualBox

Use it for convenience as vagrant does to share the puppet configuration files and applying it on the VM, once the VM has started. Its is known issue that VirtualBox shared folder performance degrades quickly as the number of files in the shared folder increases.

For any heavy duty lifting, use other mechanisms to store files on regular directories in your VM. Use NFS shared folder.

Answer 3

I had the same issue running a CentOS 7 VM with VirtualBox on a Windows 10 host: browsersync was not able to detect changes in the VBox Shared folder. What helped me is tuning the watch process a little bit. Browsersync uses chokidar to monitor the files. Setting usePolling: true solved the problem for me. How can make this useful for your case?

• try to find a grunt pluging that uses chokidar as well. Get the watcher use polling.
• tweak the settings on your watcher.