5

I have the following problem. I have two directories which I'll refer to as dir1 and dir2. They are both populated with both files and subdirectories. They are both on the same filesystem. The contents of dir1 is old, and must be replaced with the contents of dir2. The contents of dir1 can be discarded. However, I need this replacing to happen atomically. My current solution is the following.

mv dir1 dir1_backup; mv dir2 dir1; rm -rf dir1_backup

If the first command fails, the second command will not run. This is good. However, what if the second command fails? dir1 does not exist anymore, and I'm left in an invalid state.

What is the best way to accomplish this? Is it possible?

John
  • 706
  • 2
  • 10
  • 16
  • 1
    It might be too late - the easy way to do atomic directory swaps is via symbolic link. – Carl Norum Jun 14 '13 at 16:44
  • possible duplicate of [Moving a directory atomically](http://stackoverflow.com/questions/307437/moving-a-directory-atomically) – Carl Norum Jun 14 '13 at 16:45
  • don't you mean `&&` instead of `;` to separate your cmds? Unfortunately, while `&&` should work, I'm not sure I'd ever want to "bet the store" like that. I would build something that does validations, especially before the final `rm` step. Good luck. – shellter Jun 14 '13 at 17:16
  • I have seen - http://stackoverflow.com/questions/307437/moving-a-directory-atomically - before. The solutions presented within it fail my edge case. – John Jun 14 '13 at 18:45
  • Would care to explain why using symlinks, and doing a `mv -T point_to_dir2 point_to_dir1` wouldn't work in your edge case. – Fredrik Wendt May 25 '16 at 17:13

3 Answers3

3

Linux provides the renameat2 system call which with the RENAME_EXCHANGE flag will swap 2 objects:

renameat2(AT_FDCWD, "dir1", AT_FDCWD, "dir2", RENAME_EXCHANGE);
Abhyudaya Sharma
  • 1,173
  • 12
  • 18
Timothy Baldwin
  • 3,551
  • 1
  • 14
  • 23
  • If someone wants a wrapper executable on `renameat2`, I wrote one. You can download it from https://github.com/AbhyudayaSharma/exchange – Abhyudaya Sharma May 26 '21 at 08:08
1

I don't think that your assertion above that the second command will fail if the first fails is true. If mv dir1 dir1_backup fails, then dir1 will still exist, and mv dir2 dir1 will make dir2 a subdirectory of dir1. Also, since you are passing -f to rm in the third command, it will silently fail to delete a non-existant directory.

For reference, three bash operators (assuming bash here) and what they do:

  • ; simply executes the following command after the first exits, regardless of the exit status.
  • && executes the following command only if the previous command exited cleanly (ie, had an exit status of 0.)
  • || executes the following command only if the previous command exited uncleanly (ie, had a non-zero exit code.)

If you want each command's execution to be dependent upon the previous command's successful execution, then you might want to consider something like the following:

    mv dir1 dir1_backup && mv dir2 dir1 && rm -rf dir1_backup

If the moving of a directory is a failure mode you expect, then it may be reasonable to test the return value of the move or to test for the existence of the expected directory before proceeding to remove the old directory contents. If the test fails, then move the old directory contents back. The overall operation will have failed, but at least you will not be in an invalid state. For example:

    mv dir1 dir1_backup && \
    mv dir2 dir1 && \
    rm -rf dir1_backup || mv dir1_backup dir1

Since we know that && executes the following command only if the previous command exits successfully, mv dir2 dir1 will only execute if mv dir1 dir1_backup succeeds.

Further, rm -rf dir1_backup will only execute if the move of dir2 to dir1 succeeds. However, if the last exit code (the code returned by mv dir2 dir1 if it has failed) is non-zero, the || operator causes mv dir1_backup dir1 to execute.

Hope that helps! Let me know if it needs clarification.

cfunk
  • 19
  • 2
  • cfunk, thanks for clearing that up. I thought that ; and && did the opposite of what they actually do. Your solution is closer to what I want, but there is a corner case. First command completes successfully. Second command fails, triggering the last command. (mv dir1_backup dir1) Imagine if the LAST command fails. I am still left with in invalid state: dir1 does not exist. – John Jun 14 '13 at 18:43
  • If you are able to rename the directory in the first case, what is the failure mode you expect to prevent you from moving it back? If that's expected, you might want to investigate the use of symlinks instead of moving directories around. If possible, it might be best to have the application accessing the directory look at the new location and not worry about shuffling directories at all. – cfunk Jun 14 '13 at 20:52
0

On Linux you might be able to get away with bind mounting the new directory on top of the old. However you would still have issues with processes that had an open handle on a directory inside the old directory(eg their current working directory).

mount --bind dir2 dir1

Similar tricks would allow you to gain access to the old directory for cleanup purposes.

If this filesystem is exported over NFS then you would probably need to ensure the export options were set to allow traversing file system boundaries and do the bind mount on the server.

Other *nix have similar features but they aren't standardised.

William Hay
  • 2,148
  • 16
  • 20