1

The question:

How to copy one folder to another in NodeJS, overwriting only files that differ?

About fs-extra "copy" method:

It seems that the copy method from fs-extra does not have the option to skip identical files (that have not undergone any modification).

There is the overwrite option, but it only gives you the choice to overwrite in all cases, even when the files are identical, or to not overwrite, even if the files differ.

Paulo Coghi
  • 13,724
  • 14
  • 68
  • 90

3 Answers3

1

Outdated answer. See the Reflect answer instead.

I found mattijs/node-rsync and jedrichards/rsyncwrapper, both performing well and running in Linux/Windows*/etc.

For ease and freedom of choice, I decided to stick with the second option.

Examples (there are more options described in the readme's)

Single file:

rsync({
    src: "file.txt",
    dest: "tmp/file.txt"
},function (error,stdout,stderr,cmd) {
    if ( error ) {
        // failed
        console.log(error.message);
    } else {
        // success
    }
});

Full directory:

rsync({
    src: "src-folder/",
    dest: "dest-folder",
    recursive: true,
    exclude: ["*.txt"]
},function (error,stdout,stderr,cmd) {
    if ( error ) {
        // failed
        console.log(error.message);
    } else {
        // success
    }
});
Paulo Coghi
  • 13,724
  • 14
  • 68
  • 90
1

I created an open source nodejs library, called Reflect - https://github.com/alumna/reflect

I was unhappy with the fact that both libraries I mentioned earlier depends on rsync command, which is not a multi-platform decision, since Windows doesn't provide rsync natively.

I made Reflect in pure javascript to solve this, with no dependency on rsync or any other library, making it suitable for all operational systems.

Just a note: Reflect compares files similarly to rsync and syncs only the difference between two folders, completely avoiding the copy of non-modified files (or modified but still identical files), but it is still not suitable for projects with large files, since it copies the whole file when it differs.

Install

$ npm install @alumna/reflect

Usage

import reflect from '@alumna/reflect';

let { res, err } = await reflect({

    src: 'src/',
    
    dest: 'dest/',
    
    // (OPTIONAL) Default to 'true'
    recursive: true,
    
    // (OPTIONAL) Default to 'true'
    // Delete in dest the non-existent files in src
    delete: true,
    
    // (OPTIONAL)
    // Array with files and folders not to reflect
    exclude: [ "skip-this-file.txt", "skip/this/directory" ]
    
});

if ( err )
    console.error( err )

else
    console.log( res ) // Directory "src/" reflected to "dest/"
Paulo Coghi
  • 13,724
  • 14
  • 68
  • 90
0

I'd clarify here the distinction between actions "copy/merge a folder" and "sync/mirror a folder".

In case of two random folders, there is no way of knowing if two files are the same without reading their content, things like modify time and size of a file might be the same by coincidence. This is why fs-extra doesn't have such overwrite option.

If two folders are intended to be their mirrors, then you might be more optimistic about state of the world inside them and speed things up by comparing files' modify time and copying only those that changed recently. But still by design this solution can't be 100% bullet proof. Just so you know :)

Jakub Szwacz
  • 51
  • 2
  • 1
    I agree with you, and maybe it's important that I insert an observation on the README that `reflect` is currently only designed to be a mirroring library, not a merging one. – Paulo Coghi Dec 03 '21 at 11:49
  • When mirroring, it's expected that only the more recent file will prevail between two files with the same name, on the same "relative" path. But I understand that this may not be desired when "merging". – Paulo Coghi Dec 03 '21 at 12:03