2

I've got a JSON object which looks like this:

{UID_1:{
    jumpboxes:[jump_ip1, jump_ip2,...],
    hosts: [host_ip1, host_ip2,...],
    ...},
UID_2:{...

The authentication to the jumpboxes is via kerberos (passwordless), the authentication to the hosts is with password and the hosts are only visible via the jump hosts. I don't know out of the list of IPs which ones work, which are stuck, or non-responding, etc. so I need to find the first path that would let me open an SSH session.

What I can do is check for the exit codes when ssh-ing to the jump hosts with something like this:

jumpip = ''
for i in json[uid][jumpboxes]:
    if os.system('ssh {}@{}'.format(username,i))>0:
        continue
    else:
        jumpip = i
        break

This gives me the first working jumpbox ip without issues, however having a password to establish a ssh connection with the second host isn't as easy to check for the exit code of.

There're multiple ways to open the tunnel - either with os.system() and using sshpass with a session proxy (something like: if os.system('sshpass -p {} ssh -o ProxyCommand="ssh {}@{} nc {} 22" {}@{} -t {}'.format(password, user, jumpip, hosts[j], user, hosts[j], remote_cmd))>0:.... (for context let's assume the sshpass command will look something like this: sshpass -p Password123! -o ProxyCommand="ssh user@jumpbox nc hostip 22" user@hostip -t ll or doing pint in a subshell with something like os.system('ssh user@jumpbox -t ping {} -c 5'.format(hosts[j])) and although ping would return an exit code, ICMP echo replies don't mean I'd be able to open a tunnel (e.g the daemon can be stuck or could have crashed, etc.), or I can do a try-except-else block, that tries to open an ssh session to the remote host via the jumpbox with pexpect or with subrpocess.popen and with piping the stdio thus allowing me to push the password and if that fails to raise a custom exception, but I can't figure out how to get the exitcode from the ssh client, so I can check for the status...

Neither of these is robust enough for me, so I'd rather iterate through the IPs correctly, for which I'm open for suggestions.

A little bit of background - the tunnel would be used to start a nohup-ed command and then will be closed. The script uses multiprocessing and pool to go through a whole bunch of these, so I'll start them and then have a loop to check their status and retrieve the result of the remote script executed on the hosts. I know os.system is deprecated and I should use subprocess, but this isn't essential for the use-case so I don't really care about this. I'm looking for a smart way how to iterate through the possible paths which will take given a list with jumpbox with length n and a list with hosts with length m and timeout x seconds max of n*m*x seconds to figure out and instead shorten that time.

I'm also using pexpect(which uses paramiko itself) for the interactions with the remote hosts, once I've found the correct IPs I need to open the tunnel with.

Thanks in advance!

vlex
  • 131
  • 12
  • Once you get done building all that glue, you'll be close to a basic implementation of a tool something along the lines of ansible. Why not just use that? – erik258 Jan 14 '19 at 16:11
  • In this particular case the access model prevents me from using tools like ansible, puppet or chef. Otherwise I wouldn't have spent so much time trying to reinvent the wheel. Thanks anyways for the input. – vlex Jan 14 '19 at 17:13

2 Answers2

0

Paramiko's exit_status_ready function will tell you the exit status.

Return true if the remote process has exited and returned an exit status. You may use this to poll the process status if you don’t want to block in recv_exit_status. Note that the server may not return an exit status in some cases (like bad servers).

Looking at the source code for pexpect, I don't see where it uses Paramiko, so you may need to replace all of your pexpect code with Paramiko code. Paramiko gives you a lot of control over all of the low level aspects of establishing an SSH connection, so it can be a little rough to figure out, but it does give you a lot of control over the entire process.

hostingutilities.com
  • 8,894
  • 3
  • 41
  • 51
  • Hmm, I might be thinking of fabric, if it uses paramiko.. I'll have a better look at paramiko then. Thanks! – vlex Jan 14 '19 at 17:15
0

I figured it out - pexpect offers an exit code if there's a prompt, i.e. I did something along the lines of

host = ''
for i in hosts:
    cmd = 'ssh {}@{} -t ssh {}'.format(user,jumpbox, i)
    try:
        p = pexpect.spawn(cmd)
        if p.expect('.*') == 0:
            host = i
            break
    except:
        someException()
if host != '':
    ...

Thanks for all the input.

vlex
  • 131
  • 12