44

I have the following directory structure

/symdir
  sym1 -> ../dir1
  sym2 -> ../dir2
  hello.txt

And then

/dir1
  some
  files
  here
/dir2
  more
  files

I would like to replace the symlinks in symdir (sym1, sym2) with the originals. I.e.

some_awesome_bash_func symdir symdir_output

Would create 

/symdir_output
  /dir1
    some
    files
    here
  /dir2
    more
    files
  hello.txt

How would I accomplish this?

Ken Hirakawa
  • 7,831
  • 10
  • 38
  • 49

8 Answers8

72

My very personal trick for files (not directories):

sed -i '' **/*

Note that I'm using ** which uses the bash globstar option, you may have to enable it beforehand:

shopt -s globstar

How it works

I trick sed to do the job, by using an implementation detail of the sed inplace mode.

sed is a tool to edit streams of text. The -i option of sed means inplace, the empty string '' is the instruction set: so there's no instruction, sed will do nothing. **/* is a bash globstar pattern meaning "all files and all folders, at all depth, from here".

The algorithm sed uses to edit a file inplace is:

  • Create a temporary file as the output file,
  • for each line in the input file:
    • apply the transformation, write to the output file.
  • Move the output file over the input file.

As I'm asking no transformations (the empty string), the algorithm can be simplified as:

  • Create a temporary file,
  • copy the content of the original file to the temporary file
  • move the temporary file over the original file.

The temporary file is a real file, sed completly ignores that the input file was a symlink, it just reads it. So at the last step, when sed moves the temporary file over the real file, it "overwrite" the symlink with a real file, that's what we wanted.

This also explains why it won't work to transform a "symlink to a directory" to a real directory: sed works on file contents.

Julien Palard
  • 8,736
  • 2
  • 37
  • 44
  • 4
    What does it do ? How could `sed` possibly copy files ? – Nikana Reklawyks Nov 07 '12 at 23:42
  • 3
    Very interesting. I'm a bit surprised this works. (cc @LeVieuxGildas - it *does* work) – Kevin Nov 08 '12 at 00:31
  • @nikana-reklawyks sed -i FILE work like 'TMP=$(mktemp); cat $FILE | sed ... > $TMP; mv $TMP $FILE;' so without the --follow-symlinks option it will effectively do the trick :) – Julien Palard Jan 27 '13 at 16:43
  • 2
    @ZacharyVance If you have a better / cleaner / whatever solution, feel free to post it ;-) – Julien Palard Jan 19 '15 at 13:46
  • 1
    `sed -i '' **/*` by itself did not work but specifying the files more explicitly it worked: `sed -i '' *.txt` – George Pligoropoulos Nov 08 '17 at 09:10
  • 1
    @GeorgePligor You're probably missing the globstar option, I specify this in my answer. – Julien Palard Nov 08 '17 at 13:45
  • 2
    ELI5: `**/*` expands all files in directory to the command `sed -i '' `changes nothing in the content of the file This takes advantage of the fact sed uses temp files and replace the origin, looks like it works well for directories too. source: https://www.reddit.com/r/programming/comments/9ktb2o/using_sed_to_replace_symlinks_with_a_copy_of_the/ – knowledge_is_power Oct 03 '18 at 10:20
  • 1
    but sadly it has a problem with directories that are symbolic links... sed: couldn't edit : not a regular file – Manticore Oct 19 '18 at 07:45
  • 1
    @Manticore you're right, never had the issue but worth mentionning. – Julien Palard Oct 20 '18 at 08:15
  • This _is_ a cool hack, but such an answer should really, _really_ contain an explanation for how/why/when it works. – leftaroundabout Oct 21 '19 at 09:54
  • @leftaroundabout good idea, is it better? Anything I can enhance? – Julien Palard Oct 22 '19 at 12:11
  • Note that you need GNU `sed` for this to work. This does not work with FreeBSD `sed` such as the one shipped with macOS. – Atemu Dec 16 '22 at 14:04
34

You can do this easily with rsync:

rsync symdir/ symdir_output/ -a --copy-links -v

(-a means preserve basically every detail about the files, --copy-links overrides -a to turn symlinks into the real files/directories, and -v is for verbose)

Edit:

Sorry, my solution doesn't do exactly what you asked for. It will preserve the symlink's names instead of using the destination names. symdir_output would have sym1 and sym2 instead of dir1 and dir2 (though sym1 and sym2 would be a real copy of dir1 and dir2). Hope it still works for you.

Jonathan Amend
  • 12,715
  • 3
  • 22
  • 29
17

a related answer, this solution keeps the file at it's original place and creates a copy in place of the symlink

#!/bin/bash

for f in $(find . -maxdepth 1 -type l)
do
    cp --remove-destination $(readlink -e $f) $f
done
InsOp
  • 2,425
  • 3
  • 27
  • 42
PinkFloyd
  • 2,103
  • 4
  • 28
  • 47
  • 2
    This solution (replacing 'cp' with 'mv' and removing '--remove-destination') was the only one that worked exactly as I expected. It replaces the symlinks with the original files and deletes the old location. – Victor Mota Nov 17 '15 at 20:31
  • 2
    You'll want to double-quote those parameter expansions; also use `readlink -f` to if the directory is anything other than `./`: `for f in $(find . -maxdepth 1 -type l) do cp --remove-destination "$(readlink -f "$f")" "$f" done`. And be aware that splitting of the loop arguments is suspect... – Toby Speight Sep 28 '16 at 10:21
  • 1
    I'd recommend using `readlink -e` rather than plain `readlink`. Plain `readlink` will return the path the symlink points to, even if there's no file there. Better to fail on the `readlink` step than the `cp` step. – Chris Johnson Dec 15 '17 at 22:55
16

Probably not the best way, but it works:

#!/usr/bin/bash

for link in $(find /symdir -type l)
do
  loc="$(dirname "$link")"
  dir="$(readlink "$link")"
  mv "$dir" "$loc"
  rm "$link"
done
graywolf
  • 7,092
  • 7
  • 53
  • 77
Daniel Haley
  • 51,389
  • 6
  • 69
  • 95
  • 5
    `readlink "$link"` gives the next link... `readlink -e "$link"` gives the final target of a chain of links. – Peter.O Aug 23 '11 at 23:29
  • 5
    Be aware that you must not have any file name with spaces; otherwise the `mv` command breakes – Alessandro Cuttin Aug 19 '13 at 16:03
  • This doesn't "work" - it will simply move the linked files to the directory in which the link was found. What if the linked files are named differently to their links? – ventolin May 21 '23 at 09:22
7

tl;dr: much more general, much more reliable answer:

find -type l -exec sh -c 'PREV=$(realpath -- "$1") && rm -- "$1" && cp -ar -- "$PREV" "$1"' resolver {} \;

The "rsync-to-other-destination" approach is strictly superior, and usually leads to better design.

The answer by @PinkFloyd doesn't quite work with unusual filenames, "buried" symlinks, or symlinked directories. I came here because I wanted to resolve directory-symlinks, so I expect others to find this question for this reason, too. Also, my version of cp (GNU coreutils 8.25) doesn't properly handle --remove-destinationfor @PinkFloyd's answer to work with directories. So this answer uses manual rm.

Also note:

  • absence of -rf. That's because a symlink is not a directory, and should not need -r. And unless you have symlinks with restricted permissions (why would you ever want that?!), you don't need -f either.
  • realpath is perfectly fine in this context, because it allows us to find out the actual location in the current system, in the current context, and nothing else matters. This path won't be written to disk, so this is not an error.
  • the resolver string is for sh. See man sh.
  • the double-dashes everywhere are necessary, in case some symlink is called --version or similar.
  • due to the preorder-guarantee of find ("parent directory is listed at somewhere before its content"), this would first replace the parent directory, and then any symlinks within the symlinked directory. So it would work perfectly fine with "stacked" symlinks.
Mike Gebirge
  • 572
  • 7
  • 10
  • @Madeo I guess you're using some weird implementation of "find" that doesn't implement "-type"; perhaps you're using MacOS? Use one of the plenty other answers, for example the rsync one as I recommend. Or install GNU find, because you will run into similar problems again. "macos homebrew gnu find" seems to be a good search string. – Mike Gebirge Mar 07 '21 at 12:46
2

Extending Julien's suggestion. This command recursively replaces all symbolic links in the current directory with it's target.

find . -type l -exec sed -i '' {} \;
  • find .: Search in the current directory, recursively
  • -type l: Finds symlinks only
  • -exec sed -i '' {} \;: Run sed -i '' PATH for each symlink. Please see Julien's answer for an excellent explanation of why we use sed here.
thisisatotallyrandomname
  • 23
  • 4
0

I did it this way:

ls -la | awk '/-\>/{system("rm "$10); system("cp "$12" .")}'

How it works:

ls -la outputs something like this:

lrwxr-xr-x  1 username  groupname   44 10 Oct 12:17 Queue.swift -> ../../../Platform/DataStructures/Queue.swift

Column 10 is Queue.swift which is the name of the local file.
Column 12 is ../../../Platform/DataStructures/Queue.swift which is the name of the link target

The first part of the awk command is '/-\>/' which means "match lines which contain -> using a regex

The next part of the awk command is two calls to system

First system("rm "$10) which expands to system("rm Queue.swift").
This will cause the original file (the symlink) to get deleted

Second is system("cp "$12" .") which expands to system("cp ../../../Platform/DataStructures/Queue.swift .")

Putting it all together, what happens is for each file (which is a symlink), first we delete the symlink, then we copy the target file in it's place.

Although it's not part of the original question, I was using this in conjunction with git. If you happen to be doing that too, you can run git status . afterwards and you should see a bunch of type changes (and nothing else), like this:

typechange: Queue.swift
Orion Edwards
  • 121,657
  • 64
  • 239
  • 328
  • -1: That's a very fragile solution using several bad practices. Never *ever* parse the output `ls`, it fails for unusual filenames (in your case even spaces will break it). Use `find` for that. And don't trust `ls -l` columns: the date/time format for example depends on locale and is user-configurable, so it might contain fewer or more spaces. Use `readlink` to get a link's target. Also, don't use `awk` to issue system commands, use command substitution and/or `while read` loops (with `< <(...)`, `$(...)`, etc) – MestreLion Jul 08 '20 at 00:47
0

here is a slightly more general solution, based on @Daniel Haley

it also preserves the symlinks for reference and asks the user to select a directory to edit.

ls
read -p 'Which directory do you want to update?: ' linkdir

pushd $linkdir

for linkname in $(find ./ -type l)
do
  orig=$(readlink $linkname)
  mv $linkname ${linkname}.linkbak
  cp $orig $linkname
done

popd
FGiorlando
  • 1,121
  • 2
  • 12
  • 22