Skip over intermediate server via SSH

Question

I just got hired to work in a lab with a big 'ol linux cluster. I'm a mac/windows guy, so I'm still trying to figure out how to navigate around using a 'nix command line. So far I haven't run into any actual problems, but I have encountered about 5000 different little annoying things related to learning a new system.

The biggest annoyance so far is this: for whatever reason (security, something about how these servers are set up that I don't know enough about yet, etc.) when logging in to the cluster from off campus via SSH, you first have to SSH onto an old imac that's sitting on a desk in the lab, and then from there SSH into the cluster. I've been using the SCP command to copy files from my home mac to the cluster:

first in one terminal window:

 my-home-mac:scp file userA@XXX.XXX.XXX.XXX:~/target

then in another terminal window (on which I'm logged in to the lab imac):

 lab-imac:scp file userB@YYY.YYY.YYY.YYY:~/target

and the issue is that every time I do that, I'm forced to first SCP the files onto the lab imac (and enter one password), and then SCP the files from there onto the cluster (and enter another password). I'm trying to help develop this C program that's split into 4 dozen files, so I'm moving a lot of files around, and it's driving me nuts.

I guess that I'm going to have to learn shell script, but in the meantime, just to get up and running, I have a few questions:

1. how can I set up the terminal on my home mac so that I don't have to always enter the passwords for the lab imac or cluster?

2. Is there any way to SCP (or some other command) files directly from my home mac to the cluster without going through the intermediate?

3. Is there some way to set up Fugu (or any other SSH client with a gui, for that matter) on my home mac so that it can take into account the intermediate server (the lab imac) and act like it's connecting me directly to the cluster?

I'm a physicist, and everyone else in the lab are biologists, and the guy who set all of this stuff up is long gone, so I'm reaching out to you guys, all of you in the wonderful stack overflow community, to give me a hand here. Thanks.

Answer 1

In SSH, you can configure intermediate nodes to be jumped to automatically with the Host and ProxyCommand entries, see this

For not needing to enter your password, see this link

Answer 2

You can set up keys to avoid having to type passwords. I'm sure that's been covered around here before (see How do I use an SSH public key from a remote machine?; hint ssh-keygen), and I always use the OpenSSH commandline client (installed with fink on my mac, with the local package manager on linux), so I won't address silly quasi-graphical clients.

Recipe for forwarding ssh or scp through a gateway

You'll need two command lines on you original machine (which I will call orig). The gateway is called gate, and the destination dest.

First command line:

orig$ ssh -L 1111:dest.tld:22 gate.tld

This links port 1111 outgoing on orig to port 22 incoming on dest by way of a ssh tunnel through gate.

Now, if you're going to need this forwarding for a while, set this to doing something that won't let the session time out. I use top.

Second command line:

orig$ ssh -p 1111 username_on_dest@localhost

or

orig$ scp -P 1111 path/to/file/on/orig/filename username_on_dest@localhost:path/on/dest/new_filename

(notice that the port specifying option takes difrerent capitalization for ssh and scp...arghhh!) Read the documentation for various ways you can fiddle with this, but that's the basic scheme. Very useful if you use regularly resources that are kept off the public net for security reasons.

Answer 3

To avoid typing in passwords all over the place, use SSH keys. There are lots of resources here on SU and elsewhere on how to do that. One word of advice: always put a passphrase on your key!

To "bypass" the gateway mac and access the lab machines directly you need to use SSH's tunneling capabilities. You establish a link between your home Mac and the lab's gateway Mac, forwarding ports on your local mac to machines in the lab. Then you connect to your local Mac's machine on those specific ports and the traffic is forwarded to the lab machine instead.

Given the following:

• LABIMAC = the gateway mac
• LAB01 = server #1 in the lab behind the gateway
• LAB02 = server #2 in the lab behind the gateway
• LABNN = server #N in the lab behind the gateway

You could have an SSH command line that looked like the following:

ssh -L 2001:LAB01:22 -L 2002:LAB02:22 2003:LABNN:22 joe@LABIMAC

to establish the tunnels, and then do individual SSH commands to the lab machines:

ssh -p 2001 joe@localhost
ssh -p 2002 joe@localhost
ssh -p 2003 joe@localhost

The usernames can be different on each lab machine, of course. This can all get a bit wordy, so you can create aliases in the SSH config file to help:

host LAB01
hostname localhost
port 2001

host LAB02
hostname localhost
port 2002

host LAB03
hostname localhost
port 2003

Then you can skip the port and localhost on the commandline and do something more natural, once the tunnel to the lab's Mac is established:

ssh joe@LAB01

You can even ease the establishment of the tunnel by using the config file:

host LABIMAC
LocalForward 2001 LAB01:22
LocalForward 2002 LAB02:22
LocalForward 2003 LABNN:22

So that every time you ssh to LABIMAC these local port forwards are established automatically:

ssh joe@LABIMAC

(and in a second window)

ssh joe@LAB01

You can also look into using autossh to keep the initial tunneling connection open during periods of inactivity.

Answer 4

1) Already answered: SSH Keys - Auto login - No password & More secure.

|||||||

2) Transfer your files with rsync:

(i.e.) Using port 22:

rsync -av -e ssh /var/files/mysource/ root@68.x2x.27.1xx:/var/waytogo/example

Using a different port (recommended)

rsync -av -e 'ssh -p 34xx' /var/files/mysource/ root@68.x2x.27.1xx:/var/waytogo/example

^^^

The above example will not delete extra files left on the target server.

||||||||||||

3a) SSH clients don't really need a GUI, PuTTy is easy and fun after a small learning curve. :o)

3b) Forward on login could be scripted; For semi-automatic just make a command alias, and you are in with one or two keystrokes.

Note: aliases are very easy to learn, in Linux edit /root/.bashrc

alias job='ssh -p 34xx root@68.x2x.27.1xx'

Now just type " job " as a command, and you are logged into the other machine (using your SSH keys)

Answer 5

One further improvement to replace the second command is to use FileZilla as a GUI to drag between local and remote machines, assuming you have already setup the tunneling

orig$ ssh -L 1111:dest.tld:22 gate.tld

This can enable you to upload files.