Monitor remote changes and automatically copy them locally

Question

I've got a Node.js project that's local to our client machines, and I'd like to be able to monitor a remote location for changes to two files (server.js & logging.js), automatically stop the node server, copy the remotely changed file(s) locally and restart the server using the newly copied local files.

This would be used in our production environment to deploy end-of-sprint tested changes so that we don't have to regularly go run installs for our Service department.

I've tried using Forever & Nodemon, both of which happily monitor the remote file(s) for changes, but only restart the local (unchanged) copy, so those changes don't get propagated to the local copy.

I plan to use Forever in the production environment to help with the occasional local server crash we experience, so ideally any solution could be run via Forever's -c switch. I've also investigated using their underlying code chokidar, but didn't see the ability to copy files on change.

Our deployment environment is all Windows based, so I'm using Forever-Win, however, we're trying to be platform agnostic with the application we're running.

Possibly something similar to:

forever start --watch \\server\share\server.js --watch \\server\share\logging.js -c [remote monitoring copy locally package name] server.js

I'm hoping to find an NPM package that already exists so I don't have to roll my own.

This SO answer is close to what I'm looking for, but he's monitoring a log file, not the server itself.

This is another SO question that's very similar to the problem I want to solve.

score 1 · Accepted Answer · answered Mar 14 '17 at 16:52

We ended up giving up on using Forever, I couldn't get it to work properly with Nodemon and watch a remote directory at the same time. We decided it was more important to be able to push code changes on command than restart the server when it crashes (which doesn't happen very often at all). When we do a code push the server gets restarted in any case.

I created a server monitoring file called serverMon.js which contains (see Citations below for links to code I modified to my own uses):

const fs = require('fs');
const child_process = require('child_process');

//production path
var widgetPath = '\\\\server\\share\\sbSerialWidget\\';

var widgetFiles = ['sbNodeLog.js', 'server.js'];
var passedInFileName, infile, outfile;

for(var i = 0; i < widgetFiles.length; i++){
    fs.createReadStream(widgetPath + widgetFiles[i]).pipe(fs.createWriteStream(widgetFiles[i]));
}
//spawn server.js passing it's stdio, stderr, stdout back through this node instance
server = child_process.spawn('node', ['server.js'], {stdio: 'inherit'}, function (error, stdout, stderr) {
    if (error) {
        console.log(error.stack);
        console.log('Error code: ' + error.code);
        console.log('Server.js error received: ' + error.signal);
    }   
    console.log('Server.js STDOUT: ' + stdout);
    console.log('Server.js STDERR: ' + stderr);  
});

server.on('exit', function (code) {
    server.kill('SIGTERM');
    console.log('Child process exited with exit code '+code);
});

This is watched by a nodemon script contained in a .VBS file:

CreateObject("Wscript.Shell").Run "nodemon serverMon.js --exitcrash --watch \\server\share\sbSerialWidget", 0, True

Which in turn is run by a CMD script placed in the %ProgramData%\Microsoft\Windows\Start Menu\Programs\Startup so it starts the .VBS script on boot, so our little serial widget is always running in the background:

REM @echo off
REM cls
GOTO START

:START
IF EXIST "%ProgramFiles%\sbserialwidget\server.js" GOTO WIN32
IF EXIST "%ProgramFiles(x86)%\sbSerialWidget\server.js" GOTO WIN64
ECHO End start
GOTO END

:WIN32
cls
echo Inside 32 bit
PUSHD "%ProgramFiles%"\sbserialwidget
GOTO RUNVBS
GOTO END

:WIN64
echo Inside 64 bit
PUSHD "%ProgramFiles(x86)%"\sbSerialWidget
GOTO RUNVBS
GOTO END

:RUNVBS
echo Inside RUNVBS
start runNodemon.vbs
GOTO END

:END
popd
EXIT

We end up with three instances of node.exe running in Taskmanager, one running nodemon, which is monitoring the second, serverMon.js, while watching for changes on the remote files, and then the third server.js is launched as a child process of serverMon.js.

Citations:
Copying node files with streams and pipes

Spawning a node child server

score 0 · Answer 2 · answered Mar 06 '17 at 17:30

You really need two things here:

Sync your files with some remote location
Restart server when they change

There are few ways you can do it.

For example, you can host a git repo on your server and push to it from some remote place to update the code and trigger git hooks that would redeploy and restart the server.

Or you can periodically run a script on your server that would check a remote repo, update the local repo and restart the server.

Or you can use a CI service that is hooked to your changing repo, runs the tests and deploys when the tests pass.

Note that in most of those ways I'm talking about code repos, not just two files with hardcoded names, because it's much more future proof to be able to refactor your code, add files, etc. But you can also yse rsync, scp or sftp to copy the files. There are many ways to do that but it's useful to add some structure to that and use some tested tools that can scale and that are easy to maintain in the long run.

Whatever you do, just keep one thing in mind: make sure that you can verify what you are downloading before you run it. So never download anything via HTTP unless you have everything cryptographically signed. And make sure that the siggning process is secure as well. Git is very good for that because you can verify what you're downloading.

Monitor remote changes and automatically copy them locally

2 Answers2