58

I'm not sure if this is possible or not. Basically, I'm writing a script that allows me to scp a file to my hosting. This is it so far. Argument 1 is the file and argument 2 is the folder I want it to be placed in on the remote server:

function upload {
    scp $1 myusername@ssh.myhost.net:$2
}

As you may/may not know, if the directory I specify when I call the function doesn't exist, then the transfer fails. Is there a way to check if the directory exists in the function and if it doesn't, create it.

I would prefer not having to ssh in every time and create the directory, but if I have got no choice, then I have got no choice.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
larjudge
  • 1,913
  • 4
  • 14
  • 13

6 Answers6

36

You can use rsync.

For example,

rsync -ave ssh fileToCopy ssh.myhost.net:/some/nonExisting/dirToCopyTO

Note about rsync:

rsync is utility software and network protocol for Unix which synchronizes files and directories from one location to another. It minimizes data transfer sizes by using delta encoding when appropriate using the rsync algorithm which is faster than other tools.

Rahul
  • 3,293
  • 2
  • 31
  • 43
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
  • 4
    This doesn't work for me (it just complains that the destination directory does not exist) — does it require an unusually recent version of rsync? However, `rsync -R` can be used to achieve the desired effect if the local file is in an appropriately named directory, e.g: `rsync -Rave ssh some/nonExisting/dirToCopyTo/fileToCopy ssh.myhost.net:/` – Martin Kleppmann Oct 24 '12 at 23:27
  • 2
    Like @MartinKleppmann I only get an error complaining about the directory not existing using this. Have you actually witnessed this working? If so, I would be interested in figuring out what's different about our rsync programs. – DougW Jan 03 '13 at 01:19
  • 2
    Ah, looks like this works if it only has to create one level of directory to copy into. I tried deploying to /var/www/myapp/test/. When the server only had /var/www/, it failed. When I created /var/www/myapp/, it created the test folder and copied everything into it. So if you need to create more than one level of directory structure, you'll need a solution like @ire_and_curses below. – DougW Jan 03 '13 at 01:24
  • 1
    Only one level can indeed be created. So this can work `rsync fileToCopy ssh.myhost.net:some_nonexisting_dir_to_copy_to` – Pierre de LESPINAY Mar 05 '13 at 15:24
  • This does not work for me, and I saw you guys use pseudo "non existing dir" as the destination, with one level to create. Then rsync will only copy the source file and put it into the dest existing dir with the new file name. It does not create a dir. Maybe I misunderstood this, can someone please provide a real command? – HVNSweeting Jul 21 '14 at 11:39
25

I assume you mean you don't want to interactively log in and create directories by hand, rather than that you want to avoid using ssh altogether, since you still need a password or public key with scp.

If using ssh non-interactively is acceptable, then you can stream your file using cat over ssh:

cat $1 | ssh $2 "mkdir $3;cat >> $3/$1"

where

$1 = filename 
$2 = user@server
$3 = dir_on_server

If the directory already exists, mkdir complains but the file is still copied over. The existing directory will not be overwritten. If the directory does not exist, mkdir will create it.

ire_and_curses
  • 68,372
  • 23
  • 116
  • 141
  • 16
    `mkdir -p $3` avoid mkdir to complain on already existing directory. Furthermore, it let you create multiple level directories. – FabienAndre Apr 27 '11 at 12:05
19

If you do a recursive scp (-r), it will copy directories as well. So if you create a directory of the name you desire on the remote host locally, copy the file into it, and then recursively copy, the directory will be created, with the file in it.

Kind of awkward, but it would do the job.

Matthias Wandel
  • 6,383
  • 10
  • 33
  • 31
13

This is a two step process

ssh myusername@ssh.myhost.net "mkdir -p $2"

This ensures directory structure is created. Then, you copy

scp $1 myusername@ssh.myhost.net:$2
Sarang
  • 2,143
  • 24
  • 21
7

How about, for example,
ssh remote_user@remote.host '[ -d /tmp/nonexist/dir ] || mkdir -p /tmp/nonexist/dir ]'; scp test.txt remote_user@remote.host:/tmp/nonexist/dir

Rico Chen
  • 2,260
  • 16
  • 18
  • I definitely see `SSH`, as well as `scp` which, according to Wikipedia `...is based on the Secure Shell (SSH) protocol.` This isn't what the OP was asking for. – Bojangles Jul 09 '11 at 00:16
  • Nothing wrong with using ssh or scp and there's no need to avoid them. It's totally up to the user script that should remember what's been created on the remote host so the checks will not be done over and over again. I actually wrote a log backup script recently using this method. I would love to know if there are other options checking and making new folders on remote host if it doesn't provide ways such as CIFS, NFS, FTP or rsync (this one actually runs over ssh) for remote file access. – Rico Chen Jul 11 '11 at 16:23
  • Sorry if I came across the wrong way Rico, I was referring to the OP's not wanting to use SSH, but I should have read the question more carefully; scp is fine. – Bojangles Jul 11 '11 at 17:47
  • This one was the one I used. Small syntax error in the command: the `mkdir` command should not have the extra `]` on the end. – Mark D Apr 10 '23 at 23:37
0

sshfs be fancy!

Example .ssh/config

Host your-host
  HostHame example.com
  User name
  IdentitiesOnly yes
  IdentityFile ~/.ssh/private_key

Local setup aside from above only requires a target mount point...

sudo mkdir /media/your-host
sudo chown ${USER}:${USER} /media/your-host

... after which things like mounting and un-mounting are far simpler to script.

Mount

sshfs your-host:within-home/some-dir /media/your-host

Unmount

fusermount -u /media/your-host

The best part about this approach, when a server allows it, is that locally running scripts can interact with the remote file system. Meaning that things like...

if ! [ -d "/media/your-host/nowhere" ]; then
  mkdir -vp "/media/your-host/nowhere"
fi

... become possible among many other tricks that can be performed via such mounting magics.

Rahul
  • 3,293
  • 2
  • 31
  • 43
S0AndS0
  • 860
  • 1
  • 7
  • 20