1

I want to create symlinks for each file in a nested directory structure, where all symlinks will be put in one large flat folder, and have the following code by now:

# loop over directory structure:
# for all items in current directory,
# if item is directory, recurse into it;
# else it's a file, then create a symlink for it
def makelinks(folder, targetfolder, cmdprocess = None):
    if not cmdprocess:
        cmdprocess = subprocess.Popen("cmd",
                                  stdin  = subprocess.PIPE,
                                  stdout = subprocess.PIPE,
                                  stderr = subprocess.PIPE)
    print(folder)
    for name in os.listdir(folder):
        fullname = os.path.join(folder, name)
        if os.path.isdir(fullname):
            makelinks(fullname, targetfolder, cmdprocess)
        else:
            makelink(fullname, targetfolder, cmdprocess)

#for a given file, create one symlink in the target folder
def makelink(fullname, targetfolder, cmdprocess):
    linkname = os.path.join(targetfolder, re.sub(r"[\/\\\:\*\?\"\<\>\|]", "-", fullname))
    if not os.path.exists(linkname):
        try:
            os.remove(linkname)
            print("Invalid symlink removed:", linkname)
        except: pass
    if not os.path.exists(linkname):
        cmdprocess.stdin.write("mklink " + linkname + " " + fullname + "\r\n")

So this is a top-down recursion where first the folder name is printed, then the subdirectories are processed. If I run this now over some folder, the whole thing just stops after 10 or so symbolic links.

The program still seems to run but no new output is generated. It created 9 symlinks for some files in the # tag & reencode and the first three files in the ChillOutMix folder. The cmd.exe Window is still open and empty, and shows in its title bar that it is currently processing the mklink command for the third file in ChillOutMix.

I tried to insert a time.sleep(2) after each cmdprocess.stdin.write in case Python is just too fast for the cmd process, but it doesn't help.

Does anyone know what the problem might be?

Felix Dombek
  • 13,664
  • 17
  • 79
  • 131
  • This isn't really related to your problem, but have you considered using `os.walk()` rather than recursion? It'll probably be simpler. – Velociraptors Mar 10 '11 at 05:51

2 Answers2

0

Why not just execute mklink directly?

Neil
  • 54,642
  • 8
  • 60
  • 72
  • 2
    Because it's not a Windows command, it's a CMD command and needs to be run in CMD. `subprocess.call("mklink")` leads to `WindowsError: [Error 2] The system cannot find the specified file` – Felix Dombek Mar 10 '11 at 00:28
  • Sorry, I didn't realise that. The next thing I'd try would be cmd /c mklink – Neil Mar 11 '11 at 00:28
  • Of course, that's what I did, but I wanted to create several thousand symlinks and thought that this is much faster if I only use one cmd.exe process for every mklink command. It didn't work out, however; my computer just seems to close the pipe after a few commands, I have no idea why. So I finally had to do it the way you said: `subprocess.call("cmd", "/c", "mklink", linkname, filename)` for each link. And with `shell = True` or something, I even managed to suppress the cmd windows. – Felix Dombek Mar 11 '11 at 02:02
  • 1
    In that case it might be easier to use ctypes to call http://msdn.microsoft.com/en-us/library/aa363866.aspx directly. – Neil Mar 11 '11 at 23:17
0

Try this at the end:

if not os.path.exists(linkname):
    fullcmd = "mklink " + linkname + " " + fullname + "\r\n"
    print fullcmd
    cmdprocess.stdin.write(fullcmd)

See what commands it prints. You may see a problem.

It may need double quotes around mklink's arg, since it contains spaces sometimes.

Vamana
  • 580
  • 2
  • 7
  • I actually used quotes, but I tried your method nevertheless -- the commands are absolutely fine. – Felix Dombek Mar 10 '11 at 00:32
  • Maybe CMD can only take so many commands? Try feeding it something else, like `dir >>dummy.txt` repeated 100 times, see what happens. – Vamana Mar 10 '11 at 00:43