59

I need to create a symlink for every item of dir1 (file or directory) inside dir2. dir2 already exists and is not a symlink. In Bash I can easily achieve this by:

ln -s /home/guest/dir1/* /home/guest/dir2/

But in python using os.symlink I get an error:

>>> os.symlink('/home/guest/dir1/*', '/home/guest/dir2/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 17] File exist

I know I can use subprocess and run ln command. I don't want that solution.

I'm also aware that workarounds using os.walk or glob.glob are possible, but I want to know if it is possible to do this using os.symlink.

vvvvv
  • 25,404
  • 19
  • 49
  • 81
jurgenreza
  • 5,856
  • 2
  • 25
  • 37
  • 4
    I *think* `os.symlink` is a mere wrapper of the respective system call (well, more or less) and therefore wouldn't provide the same semantics you get from a full-fledged utility using that system call. – 0xC0000022L Mar 22 '13 at 21:53
  • 2
    What is the reason for the "I want to do this using `os.symlink`" requirement? If the right thing to do is a `for` loop over a `glob`, why don't you want to do the right thing? – abarnert Mar 22 '13 at 21:56
  • @abarnert I will do that if I have to. I wanted to know if there is a way to do it with `os.symlink`. I thought it would provide everything `ln -s` does, but turns out it does not. – jurgenreza Mar 22 '13 at 22:02
  • 4
    @jurgenreza: You're still missing the key point: what you're looking for isn't something `ln -s` does, it's something _the shell_ does. – abarnert Mar 22 '13 at 22:02
  • @0xC0000022L: It's a tiny bit more complicated (see [the source](5f4d7/Modules/posixmodule.c#l7199)), but only a tiny bit; basically, it ultimately calls `symlink`, `symlinkat`, or `CreateSymbolicLinkW`. And the semantics are clearly designed to match POSIX `symlink` even if that's not what it ends up using. – abarnert Mar 22 '13 at 22:04
  • @abarnert: yep, that's what my "well, more or less" was aimed at. – 0xC0000022L Mar 22 '13 at 22:05
  • @abarnert oh ok, I think I got it now. Thanks for your answer. – jurgenreza Mar 22 '13 at 22:07
  • 1
    @0xC0000022L: I wasn't correcting you, so much as taking away any doubt left by your "I _think_" prefix. – abarnert Mar 22 '13 at 22:19

3 Answers3

78

os.symlink creates a single symlink.

ln -s creates multiple symlinks (if its last argument is a directory, and there's more than one source). The Python equivalent is something like:

dst = args[-1]
for src in args[:-1]:
    os.symlink(src, os.path.join(dst, os.path.dirname(src)))

So, how does it work when you do ln -s /home/guest/dir1/* /home/guest/dir2/? Your shell makes that work, by turning the wildcard into multiple arguments. If you were to just exec the ln command with a wildcard, it would look for a single source literally named * in /home/guest/dir1/, not all files in that directory.

The Python equivalent is something like (if you don't mind mixing two levels together and ignoring a lot of other cases—tildes, env variables, command substitution, etc. that are possible at the shell):

dst = args[-1]
for srcglob in args[:-1]:
    for src in glob.glob(srcglob):
        os.symlink(src, os.path.join(dst, os.path.dirname(src)))

You can't do that with os.symlink alone—either part of it—because it doesn't do that. It's like saying "I want to do the equivalent of find . -name foo using os.walk without filtering on the name." Or, for that matter, I want to do the equivalent of ln -s /home/guest/dir1/* /home/guest/dir2/ without the shell globbing for me."

The right answer is to use glob, or fnmatch, or os.listdir plus a regex, or whatever you prefer.

Do not use os.walk, because that does a recursive filesystem walk, so it's not even close to shell * expansion.

phoenix
  • 7,988
  • 6
  • 39
  • 45
abarnert
  • 354,177
  • 51
  • 601
  • 671
  • `ln` is ln [target] [link], how can there be several targets as you describe? I think you mean several links linking to one target? – Timo May 07 '21 at 18:59
17

* is a shell extension pattern, which in your case designates "all files starting with /home/guest/dir1/".

But it's your shell's role to expand this pattern to the files it matches. Not the ln command's.

But os.symlink is not a shell, it's an OS call - hence, it doesn't support shell extension patterns. You'll have to do that work in your script.

To do so, you can use os.walk, or os.listdir. As indicated in the other answer, the appropriate call will depend on what you want to do. (os.walk wouldn't be the equivalent of *)

To convince yourself: run this command on an Unix machine in your terminal: python -c "import sys; print sys.argv" *. You'll see that it's the shell that's doing the matching.

Thomas Orozco
  • 53,284
  • 11
  • 113
  • 116
  • 1
    +1 for the experiment about the globbing. Keep in mind, however, that thw Windows shell (cmd.exe) does *not* do this before passing it to the invoked program. – 0xC0000022L Mar 22 '13 at 22:07
  • @0xC0000022L: Well, to be fair, Windows doesn't come with `ln`. And MSVCRT (their libc) comes with a hack to simulate shell globbing on `argv` either before your `main` starts, or when you explicitly ask it to, and I'm pretty sure most ports of Unix programs will either use that or something equivalent (e.g., whatever Cygwin does). – abarnert Mar 22 '13 at 22:18
  • @0xC0000022L I must confess checking whether this worked on Windows didn't cross my mind - will update! – Thomas Orozco Mar 22 '13 at 22:46
  • @abarnert: well, they call it `mklink`, just like they call what's known in the *nix world as `mkdir` simply `md` in the Windows world. – 0xC0000022L Mar 22 '13 at 22:51
  • @0xC0000022L: But `mklink` doesn't have the same syntax as `ln`. In particular, `mklink` cannot take a list of sources and link them all into a directory; it takes a single source and a single target. – abarnert Mar 22 '13 at 23:08
  • @abarnert: true point. Thought you were referring to the lack of a tool as a whole. – 0xC0000022L Mar 22 '13 at 23:13
9

As suggested by @abarnert it's the shell that recognizes * and replaces it with all the items insside dir1. Therefore I think using os.listdir is the best choice:

for item in os.listdir('/home/guest/dir1'):
    os.symlink('/home/guest/dir1/' + item, '/home/guest/dir2/' + item)
jurgenreza
  • 5,856
  • 2
  • 25
  • 37