35

I'm having a hard time copying files over to my Google Compute Engine. I am using an Ubuntu server on Google Compute Engine.

I'm doing this from my OS X terminal and I am already authorized using gcloud.

local:$ gcloud compute copy-files /Users/Bryan/Documents/Websites/gce/index.php example-instance:/var/www/html --zone us-central1-a
Warning: Permanently added '<IP>' (RSA) to the list of known hosts.
scp: /var/www/html/index.php: Permission denied
ERROR: (gcloud.compute.copy-files) [/usr/bin/scp] exited with return code [1].
Misha Brukman
  • 12,938
  • 4
  • 61
  • 78
bryan
  • 8,879
  • 18
  • 83
  • 166
  • Are you able to ssh to your instance from your local OS X machine using gcloud compute ssh command? – Faizan Jan 06 '15 at 21:51
  • @Faizan yes, I used `gcloud compute --project "" ssh --zone "us-central1-a" ""` – bryan Jan 06 '15 at 21:57
  • Does adding `sudo` to the beginning of your command make it work? – BrockLee Jan 06 '15 at 23:27
  • 3
    It seems to be an issue with the permissions on the destination directory, I think workaround would be to copy the files somewhere else maybe in /tmp and than copy them over to /var/www/html when you ssh to the instance. – Faizan Jan 07 '15 at 00:23

7 Answers7

84

insert root@ before the instance name:

local:$ gcloud compute copy-files /Users/Bryan/Documents/Websites/gce/index.php root@example-instance:/var/www/html --zone us-central1-a
Mailis Toompuu
  • 1,709
  • 15
  • 10
  • I can ssh into a VM instance, and I am using the prefix `root@` but I am still getting the same error message on `gcloud compute copy-files`. Do you have further suggestions what my help in this context? – Drux Jul 30 '16 at 19:03
  • 4
    Ah, some more evidence. I cannot `ssh` into the VM instance as root. Perhaps it's better to `scp` those files into an intermediate place like `/tmp` than trying to enable this. – Drux Jul 30 '16 at 19:10
  • This answer is contingent on how your SSH access is setup. There are many reasons not to allow root to SSH directly and, thus, this solution will not work for the case where root has been denied. The only way around that I can see is to split this into 2 steps: 1) copy files to /tmp, and 2) ssh in and use the system `sudo cp` command to copy to permanent location. – Dave Jan 03 '17 at 18:15
  • 1
    @Drux, @Dave — since this question is referring to Google Compute Engine, you cannot SSH to a VM as `root` directly, that's by design to keep your VM more secure. See [my answer](http://stackoverflow.com/a/27889822/3618671) for what to do in this case. – Misha Brukman Apr 15 '17 at 18:01
  • I will note that prefixing with `@` to `` is because without the prefix `gcloud compute copy-files` defaults to `@` , the username of the local user invoking `gcloud` on the local workstation. – Matthew Apr 20 '17 at 15:10
  • This should be in Google's documentation. Totally unclear there. – Ben Wilson Oct 05 '17 at 03:28
  • After trying a lot of things, copying to /tmp and then `mv`-ing from there is the only thing that worked for me. Thanks @Drux and @Dave ! – Anupam Jan 17 '18 at 17:41
  • thx that "root" helped - Q but how can I actually copy to "user shared 777 mode directory" ? – Bruno Jun 17 '18 at 15:50
22

The reason this doesn't work is that your username does not have permissions on the GCE VM instance and so cannot write to /var/www/html/.

Note that since this question is about Google Compute Engine VMs, you cannot SSH directly to a VM as root, nor can you copy files directly as root, for the same reason: gcloud compute scp uses scp which relies on ssh for authentication.

Possible solutions:

  1. (also suggested by Faizan in the comments) this solution will require two steps every time

    1. use gcloud compute scp --recurse to transfer files/directories where your user can write to, e.g., /tmp or /home/$USER

    2. login to the GCE VM via gcloud compute ssh or via the SSH button on the console and copy using sudo to get proper permissions:

      # note: sample command; adjust paths appropriately
      sudo cp -r $HOME/html/* /var/www/html
      
  2. this solution is one step with some prior prep work:

    1. one-time setup: give your username write access to /var/www/html directly; this can be done in several ways; here's one approach:

      # make the HTML directory owned by current user, recursively`
      sudo chown -R $USER /var/www/html
      
    2. now you can run the copy in one step:

      gcloud compute scp --recurse \
          --zone us-central1-a \
          /Users/Bryan/Documents/Websites/gce/index.php \
          example-instance:/var/www/html
      
Misha Brukman
  • 12,938
  • 4
  • 61
  • 78
4

I had the same problem and didn't get it to work using the methods suggested in the other answers. What finally worked was to explicitly send in my "user" when copying the file as indicated in the official documentation. The important part being the "USER@" in

gcloud compute scp [[USER@]INSTANCE:]SRC [[[USER@]INSTANCE:]SRC …] [[USER@]INSTANCE:]DEST

In my case I could initially transfer files by typing:

gcloud compute scp instance_name:~/file_to_copy /local_dir

but after I got the permission denied I got it working by instead typing:

gcloud compute scp my_user_name@instance_name:~/file_to_copy /local_dir

where the username in my case was the one I was logged in to Google Cloud with.

Nekroz
  • 169
  • 1
  • 1
  • 11
3

I use a bash script to copy from my local machine to writable directory on the remote GCE machine; then using ssh move the files.

SRC="/cygdrive/d/mysourcedir"
TEMP="~/incoming"
DEST="/var/my-disk1/my/target/dir"

You also need to set GCE_USER and GCE_INSTANCE

echo "=== Pushing data from $SRC to $DEST in two simple steps"
echo "=== 1) Copy to a writable temp directoy in user home"
gcloud compute copy-files "$SRC"/*.* "${GCE_USER}@${GCE_INSTANCE}:$TEMP"
echo "=== 2) Move with 'sudo' to destination"
gcloud compute ssh ${GCE_USER}@${GCE_INSTANCE} --command "sudo mv $TEMP/*.* $DEST" 

In my case I don't want to chown the target dir as this causes other problems with other scripts ...

Dr. Max Völkel
  • 1,780
  • 2
  • 17
  • 24
2

UPDATE

gcloud compute copy-files is deprecated.

Use instead:

$ gcloud compute scp example-instance:~/REMOTE-DIR ~/LOCAL-DIR \ --zone us-central1-a

More info: https://cloud.google.com/sdk/gcloud/reference/compute/scp

Victor
  • 504
  • 3
  • 9
2

The updated solution for this exact issue (2020)

For the sake of exposition, we have to break the issue in two parts. The "copy-files" command is officially depreciated and we are to use "scp", however both old and new options are limited to certain folders only.

Since we do have access to the /tmp folder, this means we can easily move our distribution files with the preferred "scp" command, as a staging step.

More importantly we also have access to execute scripts, or commands remotely via SSH on the instance which means the limited access is no longer an issue.

Example Time

The first part is to copy the dist folder, and all it's content recursively to the tmp folder to which gloud does give access:

gcloud compute scp --recurse dist user_name@instance:/tmp

The second part leverages the fact that we can run commands remotely via ssh:

gcloud compute ssh user_name@instance --command "sudo bash golive"

(or any other command you may need to execute)

More importantly this also means that we can just copy our distribution files to the final destination using sudo and the "cp" copy function:

gcloud compute ssh user_name@instance --command "sudo cp -rlf /tmp/dist/* /var/www/html/"

This completely eliminates the need to set the permissions first through the ssh terminal.

Daniel ZA
  • 306
  • 2
  • 10
-3

This is to copy files from remote machine to your machine. And make sure you have ssh setup because this will use default ssh keys. This worked for me:

gcloud compute scp 'username'@'instance_name':~/source_dir  /destination_dir --recurse

This is the generic syntax, so if you want to copy files from your machine to remote machine, you can use this. --recurse : required to copy directories with other files inside

Syntax: gcloud compute scp 'SOURCE' 'DESTINATION'

NOTE: run it without root

kumarahul
  • 396
  • 4
  • 10
  • Welcome to Stack Overflow. Before posting an answer to an older question or a question that has an accepted answer, please review answers that have already been provided and ask yourself how you will add to that existing information. In this case, the information that you've given is nothing more than command-line usage output. What's more, it's already been provided by @Nekroz, who elaborated upon why the posted content was substantive. – chb Dec 30 '18 at 19:29