rsync not synchronizing .htaccess file

Question

I am trying to rsync directory A of server1 with directory B of server2.

Sitting in the directory A of server1, I ran the following commands.

rsync -av * server2::sharename/B

but the interesting thing is, it synchronizes all files and directories except .htaccess or any hidden file in the directory A. Any hidden files within subdirectories get synchronized.

I also tried the following command:

rsync -av --include=".htaccess" * server2::sharename/B

but the results are the same.

Any ideas why hidden files of A directory are not getting synchronized and how to fix it. I am running as root user.

thanks

I've you're satisfied with one answer, you should accept it: http://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work — Kutzi, Nov 26 '15 at 10:43

Adam Zalcman · Answer 1 · 2012-01-28T16:28:12.070

118

This is due to the fact that * is by default expanded to all files in the current working directory except the files whose name starts with a dot. Thus, rsync never receives these files as arguments.

You can pass . denoting current working directory to rsync:

rsync -av . server2::sharename/B

This way rsync will look for files to transfer in the current working directory as opposed to looking for them in what * expands to.

Alternatively, you can use the following command to make * expand to all files including those which start with a dot:

shopt -s dotglob

See also shopt manpage.

edited Jan 28 '12 at 16:28

answered Jan 28 '12 at 16:22

Adam Zalcman

26,643
4
71
92

8

Indeed '.' solves the issue, e.g. `rsync -avz user@domain.com:/remote/path/. /destination/path` does the job – Mariusz Nowak Aug 20 '13 at 12:12

score 61 · Answer 2 · answered May 21 '13 at 14:41

61

For anyone who's just trying to sync directories between servers (including all hidden files) -- e.g., syncing somedirA on source-server to somedirB on a destination server -- try this:

rsync -avz -e ssh --progress user@source-server:/somedirA/ somedirB/

Note the slashes at the end of both paths. Any other syntax may lead to unexpected results!

Also, for me its easiest to perform rsync commands from the destination server, because it's easier to make sure I've got proper write access (i.e., I might need to add sudo to the command above).

Probably goes without saying, but obviously your remote user also needs read access to somedirA on your source server. :)

answered May 21 '13 at 14:41

Brian Lacy

18,785
10
55
73

Was it the -e flag that will sync the `.htaccess` file? – Jesse Burcsik Oct 01 '14 at 20:39
1

@JesseBurcsik No, the `-e ssh` specifies the method for connecting. – tripleee Apr 04 '16 at 10:53
1

You just need slash on the source, destination doesn't matter – Jonas Stensved Apr 16 '16 at 10:23

harleygolfguy · Answer 3 · 2018-04-23T22:27:27.317

34

I had the same issue.

For me when I did the following command the hidden files did not get rsync'ed

rsync -av /home/user1 server02:/home/user1

But when I added the slashes at the end of the paths, the hidden files were rsync'ed.

rsync -av /home/user1/ server02:/home/user1/

Note the slashes at the end of the paths, as Brian Lacy said the slashes are the key. I don't have the reputation to comment on his post or I would have done that.

edited Apr 23 '18 at 22:27

answered Nov 25 '15 at 19:04

harleygolfguy

805
9
14

1

Just add `shopt -s dotglob` to your script before the rsync – Pian0_M4n Jul 10 '18 at 12:24
(Over the course of years I have made it an iron rule (on many scripts and occasions): trailing slash ALWAYS on source, NEVER on dest. I even enforce this in wrapping helper routines...) – Frank N Oct 03 '21 at 17:42

vkraemer · Answer 4 · 2012-01-28T16:40:05.730

4

I think the problem is due to shell wildcard expansion. Use . instead of star.

Consider the following example directory content

$ ls -a .
. .. .htaccess a.html z.js

The shell's wildcard expansion translates the argument list that the rsync program gets from

-av * server2::sharename/B

into

-av a.html z.js server2::sharename/B

before the command starts getting executed.

edited Jan 28 '12 at 16:40

answered Jan 28 '12 at 16:27

vkraemer

9,864
2
30
44

score 3 · Answer 5 · edited Feb 26 '19 at 19:00

3

The * tell to rsynch to not synch hidden files. You should not omit it.

edited Feb 26 '19 at 19:00

Bhargav Rao

50,140
28
121
140

answered Jan 28 '12 at 16:23

DonCallisto

29,419
9
72
100

score 0 · Answer 6 · answered Sep 23 '22 at 04:08

On a related note, in case any are coming in from google etc trying to find while rsync is not copying hidden subfolders, I found one additional reason why this can happen and figured I'd pay it forward for the next guy running into the same thing: if you are using the -C option (obviously the --exclude would do it too but I figure that one's a bit easier to spot).

In my case, I had a script that was copying several folders across computers, including a directory with several git projects and I noticed that the I couldn't run any of the normal git commands in the copied repos (yes, normally one should use git clone but this was part of a larger backup that included other things). After looking at the script, I found that it was calling rsync with 7 or 8 options.

After googling didn't turn up any obvious answers, I started going through the switches one by one. After dropping the -C option, it worked correctly. In the case of the script, the -C flag appears to have been added as a mistake, likely because sftp was originally used and -C is a compression-related option under that tool.

per man rsync, the option is described as

--cvs-exclude, -C auto-ignore files in the same way CVS does

Since CVS is an older version control system, and given the man page description, it makes perfect sense that it would behave this way.

rsync not synchronizing .htaccess file

6 Answers6