Gulp RSync - Save progress to log file

Question

I've been using gulp-rsync for deploying static assets to my server via gulp task. And I use incremental option to determine whether a file needs to be updated or not. This works fine :)

I need save the progress displayed in the console in a file or whatever.. Because i need purge individual files in the cache on cloudflare :/

Looks like that (in the console):

[20:49:52] Starting 'rsync'...
[20:49:53] gulp-rsync: Starting rsync to my-ssh-hostname:/example/public-html/assets/...
[20:49:55] gulp-rsync: sending incremental file list
[20:49:57] gulp-rsync: favicon.ico
[20:49:57] gulp-rsync:         1150 100%  439.45kB/s    0:00:00 (xfer#1, to-check=12/13)
[20:49:57] gulp-rsync: css/main.css
[20:49:57] gulp-rsync:         2712 100%  101.86kB/s    0:00:00 (xfer#2, to-check=11/13)
[20:49:57] gulp-rsync: css/style.css
[20:49:57] gulp-rsync:         1445 100%   54.27kB/s    0:00:00 (xfer#3, to-check=9/13)
[20:49:57] gulp-rsync: js/app.js
[20:49:57] gulp-rsync:        31878 100%    1.09MB/s    0:00:00 (xfer#7, to-check=3/13)
[20:49:57] gulp-rsync: scripts.js
[20:50:01] gulp-rsync:        76988 100%    2.53MB/s    0:00:00 (xfer#9, to-check=1/13)
[20:50:01] gulp-rsync: sent 2401 bytes  received 2820 bytes  10442.00 bytes/sec
[20:50:02] gulp-rsync: total size is 10106  speedup is 4.37
[20:50:02] gulp-rsync: Finished 'rsync' after 3.38 s

I need save and extract the files in log:

favicon.ico,
css/main.css,
css/style.css, 
js/app.js, 
scripts.js

-- My original "gulpfile.js" :

var
  gulp = require('gulp'),
  gutil = require('gulp-util'),
  rsync = require('gulp-rsync'),
  logCapture = require('gulp-log-capture');

var config = {
  hostname : 'my-ssh-hostname',
  destination : '/example/public-html/assets/',
  progress: true,
  incremental: true,
  relative: true,
  emptyDirectories: true,
  recursive: true,
  clean: true,
  exclude: ['._', '.DS_Store' , 'thumbs.db', 'desktop.ini'],
  chmod: '775',
};

gulp.task('rsync', function (){
  return gulp.src('./my-local-dir/' + '**/*')
    .pipe(rsync(config))
});

-- I found the "Log Capture Plugin for Gulp" - gulp-log-capture

What's the right way to use? :/

gulp.task('rsync', function (){
  return gulp.src('./my-local-dir/' + '**/*')
    .pipe(logCapture.start(process.stdout, 'write'))
    .pipe(rsync(config))
    .pipe(logCapture.stop('txt'))
    .pipe(gulp.dest('./dest'));
});

Any sugestions will be apreciated :)

Answer 1

You cannot use gulp-log-capture for this because of the way that gulp-rsync works.

gulp-rsync waits until it has collected all the file names before it invokes rsync. Otherwise it would have to call rsync for each individual file which would severely degrade performance.

So by the time that rsync is invoked logCapture has already stopped capturing the log output.

You need to listen for the 'end' event and capture the log output yourself by substituting process.stdout.write() with a function of your own (which is what gulp-log-capture does):

gulp.task('rsync', function (){
  var logOutput = "";
  var stdoutWrite;
  return gulp.src('./my-local-dir/' + '**/*')
    .on('end', function() {
      stdoutWrite = process.stdout.write;
      process.stdout.write = function(output) { logOutput += output; };
    })
    .pipe(rsync(config))
    .on('end', function() {
      process.stdout.write = stdoutWrite;
      process.stdout.write(logOutput);
      fs.writeFileSync('./dest/logOutput.txt', new Buffer(logOutput));
    });
});

This stores the progress log generated by rsync in the logOutput variable and writes it to the file ./dest/logOutput.txt.

Extracting the files names from logOutput is now just a matter of coming up with the appropriate regexes. I'll leave that part to you. If this gives you trouble, it's probably best to ask the people over in regex.