How do I merge one directory into another using Bash?

Question

I'm looking for shell script that merge files from one directory into another.

Sample:

html/
  a/
    b.html
  index.html

html_new/
  a/
    b2.html
    b.html

Usage:

./mergedirs.sh html html_new

Result:

html/
  a/
    b.html
    b2.html
  index.html

html/a/b.html was replaced by html_new/a/b.html
html/a/b2.html was copied from html_new/a/b2.html
html/index.html was kept untouched

Tried `cp -r html_new/* html` ? You can remove `html_new` directory afterwards. — ismail, Dec 31 '10 at 20:46

Flimm · Answer 1 · 2019-09-05T10:38:36.560

111

cp -RT source/ destination/

All files and directories in source will end up in destination. For example, source/file1 will be copied to destination/file1.

The -T flag stops source/file1 from being copied to destination/source/file1 instead. (Unfortunately, cp on macOS does not support the -T flag.)

edited Sep 05 '19 at 10:38

answered Nov 22 '13 at 10:22

Flimm

136,138
45
251
267

works nicely, it copies and merges /folder/data to folder2/data – nwgat Jan 15 '15 at 14:33
3

I pefer this solution because it works "as is" with .dotfiles. – fnkr Sep 05 '15 at 11:12
14

On Mac OS X / darwin you'll want to use `cp -r source/ destination` -- it's tricky you have to use the / exactly as indicated. – Ron Wertlen Jul 25 '18 at 14:10
Proud to be the one upvoting this answer and so making it now equals to the current accepted one . – Hadrien TOMA Jan 07 '20 at 22:07
Please note is will overwrite any files in destination which have the same name as the source – brianegge Jan 26 '20 at 02:11

Luke Maurer · Accepted Answer · 2011-12-07T22:42:01.970

91

You probably just want cp -R $1/* $2/ — that's a recursive copy.

(If there might be hidden files (those whose names begin with a dot), you should prefix that command with shopt -s dotglob; to be sure they get matched.)

edited Dec 07 '11 at 22:42

answered Dec 31 '10 at 20:46

Luke Maurer

7,845
2
24
24

Are the trailing slashes important here? – Flimm Aug 02 '22 at 10:26

score 18 · Answer 3 · edited Aug 02 '22 at 10:41

18

Take a look at rsync

rsync --recursive html/ html_new/

Notice that the trailing slash / matters in this case. If you omit it from the source argument, rsync will write the files to html_new/html/ instead of html_new/.

Rsync has got a lot of flags to set so look at rsync manpage for details.

edited Aug 02 '22 at 10:41

Flimm

136,138
45
251
267

answered Dec 31 '10 at 21:05

CarlG

590
2
9

score 8 · Answer 4 · edited Aug 25 '18 at 07:20

8

Just use rsync - it's a great tool for local file copy and merging in addition to remote copying.

rsync -av /path/to/source_folder/ /path/to/destination_folder/

Note that the trailing slash on the source folder is necessary to copy only the contents of source_folder to the destination. If you leave it off, it will copy the source_folder and it's contents, which is probably not what you are looking for since you want to merge folders.

edited Aug 25 '18 at 07:20

Mikkel

1,192
9
22

answered Dec 23 '13 at 17:35

Gavster

131
1
1

there is small typo, extra hyphen, so it should be `rsync -av /path/to/source_folder/ /path/to/destination_folder/` – vim Jan 21 '18 at 10:33

score 6 · Answer 5 · edited Apr 13 '17 at 12:36

Even though this question and its accepted answer are ancient, I am adding my answer because the presently existing ones using cp either don't handle some edge-cases or require working interactively. Often edge-cases/scriptability/portability/multiple-sources don't matter though, in which case simplicity wins, and it is better to use cp directly with less flags (as in other answers) to reduce cognitive load - but for those other times (or for a robustly reusable function) this invocation/function is useful, and incidentally isn't bash-specific (I realise this question was about bash though, so that's just a bonus in this case). Some flags can be abbreviated (e.g. with -a), but I have included all explicitly in long-form (except for -R, see below) for the sake of explanation. Obviously just remove any flags if there is some feature you specifically don't want (or you are on a non-posix OS, or your version of cp doesn't process that flag - I tested this on GNU coreutils 8.25's cp):

mergedirs() {
    _retval=0
    _dest="$1"
    shift
    yes | \
        for _src do
            cp -R --no-dereference --preserve=all --force --one-file-system \
                  --no-target-directory "${_src}/" "$_dest" || { _retval=1; break; }
        done 2>/dev/null
    return $_retval
}

mergedirs destination source-1 [source-2 source-3 ...]

Explanation:

-R: has subtly different semantics from -r/--recursive on some systems (particularly with respect to special files in source dirs) as explained in this answer
--no-dereference: never follow symbolic links in SOURCE
--preserve=all: preserve the specified attributes (default: mode,ownership,timestamps), if possible additional attributes: context, links, xattr, all
--force: if an existing destination file cannot be opened, remove it and try again
--one-file-system: stay on this file system
--no-target-directory: treat DEST as a normal file (explained in in this answer, namely: If you do a recursive copy and the source is a directory, then cp -T copies the content of the source into the destination, rather than copying the source itself.)
[piped input from yes]: even with --force, in this particular recursive mode cp still asks before clobbering each file, so we achieve non-interactiveness by piping output from yes to it
[piped output to /dev/null]: this is to silence the messy string of questions along the lines of cp: overwrite 'xx'?
[return-val & early exit]: this ensures the loop exits as soon as there is a failed copy, and returns 1 if there was an error

BTW:

A funky new flag which I also use with this on my system is --reflink=auto for doing so-called "light copies" (copy-on-write, with the same speed benefits as hard-linking, and the same size benefits until and in inverse proportion to how much the files diverge in the future). This flag is accepted in recent GNU cp, and does more than a no-op with compatible filesystems on recent Linux kernels. YMWV-a-lot on other systems.

DVK · Answer 6 · 2010-12-31T20:59:51.033

2

Wouldn't cp -r work?

cp -r html_new/* html

or (since the first version won't copy ".something" files)

cd html_new; cp -r . ../html

Please note that -r reads from pipes if any of the files in the copied directory are pipes. To avoid that, use -R instead.

edited Dec 31 '10 at 20:59

answered Dec 31 '10 at 20:45

DVK

126,886
32
213
327

When "merging" multiple directories etc. consider also `-n` (no clobber) or `-f` (force) option to either keep existing files or overwrite them. – Tomasz Gandor May 31 '14 at 20:53

score 2 · Answer 7 · answered Dec 31 '10 at 20:45

2

cd html
cp -r . /path/to/html_new

answered Dec 31 '10 at 20:45

Brian Clapper

25,705
7
65
65

score 0 · Answer 8 · answered Jun 16 '21 at 06:19

cp --recursive --no-target-directory [--no-clobber] "[IN]" "[OUT]"

--no-target-directory: Treat OUT as a normal file, so include files starting with dot (hidden files).
--no-clobber: If you don't want to overwrite existing files. Inner dirs are never overwritten anyway.

How do I merge one directory into another using Bash?

8 Answers8

Linked