0
from os import link

link('WInd_Rose_Aguiar.svg', 'Wikipedia Daily Featured Picture')

# A day has passed

link('Piero_del_Pollaiuolo_-_Profile_Portrait_of_a_Young_Lady_-_Gemäldegalerie_Berlin_-_Google_Art_Project.jpg',
     'Wikipedia Daily Featured Picture') # Exception

The results of calling the above script:

my@comp:~/wtfdir$ python3 wtf.py
Traceback (most recent call last):
  File "wtf.py", line 8, in <module>
    'Wikipedia Daily Featured Picture') # Exception
FileExistsError: [Errno 17] File exists: 'Piero_del_Pollaiuolo_-_Profile_Portrait_of_a_Young_Lady_-_Gemäldegalerie_Berlin_-_Google_Art_Project.jpg' -> 'Wikipedia Daily Featured Picture'

Creating the first link succeeds. Creating the second one fails.

That's hardly what I would expect… My intention is to overwrite this link.

https://docs.python.org/3/library/os.html#os.link ⇐ I can’t see a force or overwrite_if_exists or similar parameter to the function link in the docs.

How can I create a link pointing to a new source, overwriting the previous link if it exists?

Well yes – I guess I can do sth like this:

from os import link, remove
from os.path import isfile

def force_link(src, dest):
    if isfile(dest):
        remove(dest)
    link(src, dest)

force_link('WInd_Rose_Aguiar.svg', 'Wikipedia Daily Featured Picture')

# A day has passed

force_link('Piero_del_Pollaiuolo_-_Profile_Portrait_of_a_Young_Lady_-_Gemäldegalerie_Berlin_-_Google_Art_Project.jpg',
     'Wikipedia Daily Featured Picture') # No longer exception

But this is cumbersome and at least in theory may be incorrect (what if some other process re-creates the file between remove(dest) and link(src, dest)?). And while perhaps this (at least theoretical) incorrectness could be resolved, the resulting code would be even more cumbersome, I guess…

There must be a better, more right-handed way to do this!

  • What do you mean by `repoint this link`? – wildplasser Nov 20 '19 at 14:26
  • Aside from your question, the `if isfile(dest): remove(dest)` is wrong. Are you really want to only remove dest if it is a file? – Marcin Orlowski Nov 20 '19 at 14:27
  • Use `os.replace`. – chepner Nov 20 '19 at 14:29
  • I think the OP is using hard links, but thinking in terms of symbolic links. (which would make more sense here, anyway) – wildplasser Nov 20 '19 at 14:33
  • @chepner From the docs (https://docs.python.org/3/library/os.html#os.replace): "*Rename the file or directory src to dst.*" But I don't want to rename a file! I want to create a link to a file, not rename it! I also don't want to rename the existing link, but to overwrite it! –  Nov 20 '19 at 14:33
  • @MarcinOrlowski Yes - it's not supposed to be a directory. If it is a directory, then something weird is going on so we may just as well raise and fail. (`remove` does remove hardlinks btw) –  Nov 20 '19 at 14:34
  • @wildplasser I'm using hardlinks because I want to be prepared for the possibility that the "original" files (`WInd_Rose_Aguiar.svg`) are moved / renamed. Doing so will break existing symlinks AFAIK, but hardlinks are (AFAIK) guaranteed to stay correct, pointing to the same file (inode). –  Nov 20 '19 at 14:39
  • @wildplasser Sorry if *repoint this link* is unclear? I want the behavior to be analogous to what my second snippet does. –  Nov 20 '19 at 14:42
  • 2
    man 2 link: `link() creates a new link (also known as a hard link) to an existing file. If newpath exists it will not be overwritten.` So, what you want is impossible, the syscall does not allowed to (re)create a link that already exists. You;ll first have to remove/unlink the copy, then create a new one. This of course creates a race-condition. [also: this is for Unix. The Windows filesystem will probably behave differntly] – wildplasser Nov 20 '19 at 14:46
  • @wildplasser Well, tough. Thank you for your answer ☺ –  Nov 20 '19 at 15:01
  • 1
    @gaazkam You can create a fresh hard link (possibly using `tempfile` to guarantee a unique name), the replace the old link with the new one. I don't quite have a good implementation worked up yet, but that approach should work. (Roughly speaking, `os.link(src, "some_temp_name"); os.replace("some_temp_name", dest)`. `link` and `replace` are atomic, so you'll just have to make sure that `some_temp_name` is only accessible to you, to avoid anyone renaming or deleting it between the two calls) – chepner Nov 20 '19 at 15:02
  • @chepner that should work. `rename` is atomic. – wildplasser Nov 20 '19 at 15:07
  • @wildplasser Yeah, the atomicity isn't what I am worried about; it's using `tempfile` correctly to make sure that an adversary couldn't mess with the new link before you have a chance to call `os.replace`. I think I have it right in the answer, but I'm not positive. – chepner Nov 20 '19 at 15:13

1 Answers1

2

Create a new link for the file you want to expose. Then replace your fixed link with the new link you just created.

from tempfile import TemporaryDirectory


def force_link(src, dest):
    with TemporaryDirectory(dir=os.path.dirname(dest)) as d:
        tmpname = os.path.join(d, "foo")
        os.link(src, tmpname)
        os.replace(tmpname, dest)

You may need to ensure that the permissions on dest are correct afterwards.

os.link will succeed in securely creating a new link in the temporary directory. Then you'll use os.replace to securely rename the temporary link to dest, effectively overwriting the old link with the new.

chepner
  • 497,756
  • 71
  • 530
  • 681
  • In theory, one could keep listing the contents of `dirname(dest)` in a brute-force way and try to immediately mess with any new file that appears there, but I guess this is as far as we can go. For all practical purposes this guarantees against accidental problems I guess, which is fine for me. –  Nov 20 '19 at 15:20
  • `NamedTemporaryFile` creates a file that only you have permission to write; that's why I mentioned need to adjust the permissions after you call `replace`. (For full security, you may need to run the process as a user which no one else can impersonate, but you are always at the mercy of anyone who has root access.) – chepner Nov 20 '19 at 15:24
  • Ugh... I've just run your code... `FileExistsError: [Errno 17] File exists: 'WInd_Rose_Aguiar.svg' -> '/home/my/wtfdir/tmpup2vyro4'` Are you sure we can write `os.link(src, f.name)` when `f` is a file that exists? Are we not running here in the same precise problem that one cannot create a hardlink with a name of an already existing file? –  Nov 20 '19 at 15:36
  • I wonder if the deprecated [`mktemp`](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) is not the only available option here... –  Nov 20 '19 at 15:40
  • 1
    Yeah, that's the part I overlooked. It might be simpler to create a temporary *directory*; then you can use whatever hardcoded name you want for the intermediate name because no one else will be able to create file in or delete files from that directory. – chepner Nov 20 '19 at 15:42