Persistent WinSCP connection for batch copy in Python

Question

I'm trying to copy thousands files to a remote server. These files are generated in real-time within the script. I'm working on a Windows system and need to copy the files to a Linux server (hence the escaping).

I currently have:

import os
os.system("winscp.exe /console /command  \"option batch on\" \"option confirm off\" \"open user:pass@host\" \"put f1.txt /remote/dest/\"")

I'm using Python to generate the files but need a way to persist the remote connection so that I can copy each file, to the server, as it is generated (as opposed to creating a new connection each time). That way, I'll only need to change the field in the put option thus:

"put f2 /remote/dest"
"put f3 /remote/dest"

etc.

score 5 · Answer 1 · edited Jan 15 '19 at 13:40

I needed to do this and found that code similar to this worked well:

from subprocess import Popen, PIPE

WINSCP = r'c:\<path to>\winscp.com'

class UploadFailed(Exception):
    pass

def upload_files(host, user, passwd, files):
    cmds = ['option batch abort', 'option confirm off']
    cmds.append('open sftp://{user}:{passwd}@{host}/'.format(host=host, user=user, passwd=passwd))
    cmds.append('put {} ./'.format(' '.join(files)))
    cmds.append('exit\n')
    with Popen(WINSCP, stdin=PIPE, stdout=PIPE, stderr=PIPE,
               universal_newlines=True) as winscp: #might need shell = True here
        stdout, stderr = winscp.communicate('\n'.join(cmds))
    if winscp.returncode:
        # WinSCP returns 0 for success, so upload failed
        raise UploadFailed

This is simplified (and using Python 3), but you get the idea.

score 2 · Answer 2 · answered Nov 19 '11 at 22:33

2

Instead of using an external program (winscp) you could also use an python ssh-library like pyssh.

answered Nov 19 '11 at 22:33

Nicola Coretti

2,655
2
20
22

1

so how could i accomplish my task using pyssh? – Duke Nov 19 '11 at 22:35

Martin Prikryl · Answer 3 · 2017-02-28T09:43:23.953

You would have to start persistent WinSCP sub-process in Python and feed the put commands to its standard input continuously.

I do not have Python example for this, but there's an equivalent JScript example:
https://winscp.net/eng/docs/guide_automation_advanced#inout
or C# example:
https://winscp.net/eng/docs/guide_dotnet#input

Though using WinSCP .NET assembly via its COM interface for Python would be a way easier:
https://winscp.net/eng/docs/library

Persistent WinSCP connection for batch copy in Python

3 Answers3