Rsync to Google Compute engine Instance from Jenkins

Question

I want to send files from Jenkins to to my instance in Google Compute engine instance I added a build in my config in jenkins :

rsync -vrzhe "ssh -i /var/lib/jenkins/.ssh/google_compute_engine -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no" . login@Host:/var/www

And I get this error :

Checking out Revision 59cf9dd819fe2168c4c40f716707d58b2b99e251 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 59cf9dd819fe2168c4c40f716707d58b2b99e251
> git rev-list 59cf9dd819fe2168c4c40f716707d58b2b99e251 # timeout=10
[Platform] $ /bin/sh -xe /tmp/hudson4502433356962914860.sh
+ rsync -vrzhe 'ssh -i /var/lib/jenkins/.ssh -o UserKnownHostsFile=/dev/null -o

CheckHostIP=no -o StrictHostKeyChecking=no' . login@Host:/var/www
   StrictHostKeyChecking=no' . login@host:/var/www
   ssh: connect to host host port 22: Connection timed out
   rsync: connection unexpectedly closed (0 bytes received so far) [sender]
   rsync error: unexplained error (code 255) at io.c(601) [sender=3.0.7]
   Build step 'Exécuter un script shell' marked build as failure
   Finished: FAILURE

Any idea

score 19 · Answer 1 · answered Mar 10 '18 at 00:08

first configure .ssh config file by (assuming you installed gcloud sdk):

gcloud compute config-ssh

It will tell you that "now you can use ssh/scp with your instances by running

ssh your-instance

your-instance is in the form of "instance.zone.project", usually.

Now you can rsync:

rsync -ave ssh your-local-dir your-instance:~/your-destination

Done.

You can mention the user if you like so:

rsync -ave ssh your-local-dir your-user@your-instance:~/your-destination

It worked for me. Juts do not forget to replace "your-instance" (and your-user) with the correct one. You can get it through "gcloud compute config-ssh" or "gcloud compute config-ssh --dry-run" or go to your cloud.google.com then compute engine then vm instances then from connect choose view gcloud command. All will show your instance name in the form of "instance.zone.project."

I hope it will help someone in future. :)

You could mention about the `-z` flag too which does speed up things by enabling compression in most workflows. — Sankar, Jun 27 '20 at 07:01
Do not forget to install rsync on the server! `sudo apt-get install rsync`. Otherwise it won't work. If you use the default debian image it's not installed. — Philippe Remy, Aug 13 '20 at 13:57

score 5 · Answer 2 · answered Jan 05 '18 at 00:46

5

Maybe a little late, but you can use the gcloud compute ssh command directly instead of finding the google ssh keys.

First, you have to make a script that will hide the rsync ssh command args from gcloud:

cat >./gcloud-compute-ssh <<EOF
#! /bin/sh
host="$1"
shift
exec gcloud compute ssh "$host" -- "$@"
EOF

chmod a+x ./gcloud-compute-ssh

Then you can rsync -e to your heart's content:

rsync -e ./gcloud-compute-ssh my-dir my-instance:/my-dir

answered Jan 05 '18 at 00:46

noelbk

1,433
1
12
8

Have you tried it by yourself? It does not work for me. – Ali Khosro Mar 08 '18 at 09:46
Of course! Can you describe what didn't work, perhaps with a terminal log? – noelbk Mar 09 '18 at 18:13
Sorry, after a little twik in my .ssh file, it started working. Thanks. – Ali Khosro Mar 09 '18 at 20:31

score 3 · Answer 3 · answered Jan 11 '15 at 18:39

3

Your rsync command above uses the -i option for ssh. The argument to -i option should be path to ssh key file, not the directory where the key file is.

answered Jan 11 '15 at 18:39

sti

11,047
1
27
27

Yes of course -i should be the path for the file i will edit my post but that doesn't solve my problem – Rjaibi Mejdi Jan 14 '15 at 13:29

Rjaibi Mejdi · Accepted Answer · 2017-04-26T10:23:49.797

1

The problem was with my public key so to solve the problem you need to :

1 - Setting up your ssh keys : run

gcloud compute ssh example-instance

2 - cp your .ssh/google_compute_engine.pub content into the GCE VM authorized_key

3 - restart VM instance

Thanks for your help

edited Apr 26 '17 at 10:23

answered Jan 15 '15 at 11:14

Rjaibi Mejdi

6,820
3
21
26

At least you didn't have any incompatibility between rsync and ssh, as I feared initially. +1 – VonC Jan 15 '15 at 12:09

score 0 · Answer 5 · edited May 23 '17 at 11:33

0

As mentioned in "How to solve rsync error: error in rsync protocol data stream (code 12) at io.c(600) on windows", check which ssh get to be actually used, and if that ssh version is compatible with with the one from rsync.

Specifying an absolute path for the ssh executable can help:

rsync -e /usr/bin/ssh ....

This thread also suggests to purge ~/.ssh/know_hosts in case the server has changed and now has a different key.

edited May 23 '17 at 11:33

Community

1
1

answered Jan 14 '15 at 13:46

VonC

1,262,500
529
4,410
5,250

@RjaibiMejdi What OS are you on?, what version of rsync and ssh are you using? – VonC Jan 14 '15 at 15:05
Ubuntu server to Ubuntu server / OpenSSH_5.5p1 / rsync version 3.0.7 protocol version 30 – Rjaibi Mejdi Jan 14 '15 at 15:17
I mean what exact precise complete version number? What does uname -a return? What rsync -v return? – VonC Jan 14 '15 at 15:19
Linux ns61740.ovh.net 3.2.13-grsec-xxxx-grs-ipv6-64 #1 SMP Thu Mar 29 09:48:59 UTC 2012 x86_64 GNU/Linux – Rjaibi Mejdi Jan 14 '15 at 15:21
Ok, the thread I mentioned in the answer seemed to suggest a potential incompatibility between certain versions of rsync and ssh, hence my previous question. – VonC Jan 14 '15 at 15:24

Rsync to Google Compute engine Instance from Jenkins

5 Answers5