7

WARNING -> Please be careful when reading this problem description. I had some assumptions that were not correct as I was writing this question. Make sure you read my answer explaining what I had wrong!

I have host A in AWS as an EC2 instance.

I a private key embedded in a .PEM file that I can use to host A with SSH key Z. This works if I pass it to the ssh command using the -l argument, AND if I turn off strict host checking with -o StrictHostKeyChecking=no.

I would strongly prefer to leave strict host checking on even though I "know" this is the correct host because I'm interacting with the AWS interface, getting the ip/dns from them, and I'm inside of my own little VPC world.

It seems like the only way to provide the fingerprint -> host mapping is by providing it in a known_hosts file.

Is that correct?

If that is correct, how can I take the private key embedded in the .PEM file that I have from AWS and build the correct entry for the single fingerprint -> host mapping for a temporary known_hosts file that I can read when I'm logging into the EC2 instance?

WHAT I DO NOT WANT TO DO

  • Use ssh-keyscan. All this does is blindly accept the fingerprint of the remote client without validating that it matches with the key. I think?
  • Turn off StrictHostKeyChecking. I want to establish good practices early, and I need to know how to do this now, because I'm going to need to know how to do this in general. (By this I mean how to use SSH fingerprints to validate the identity of the host I'm connecting to, based on the key that I have.)
  • Mess around with ssh-add. I want to write this to a file that's easy to lockdown access to, not put it into a running process.

EDITS: Strangely when I try to extract the fingerprint from the pem file it doesn't match the fingerprint I see when I connect and it prompts me.

FINGERPRINT EXTRACTION FROM PEM

bash-4.2$ ssh-keygen -l -E md5 -f ./blah.PEM
2048 MD5:be:b1:d7:e1:f0:0f:ce:41:60:fa:97:dc:b8:2c:ed:08 no comment (RSA)
bash-4.2$ ssh-keygen -l -E sha1 -f ./blah.PEM
2048 SHA1:g2PDmIcw19Z/v7HTco6xRWxQ88c no comment (RSA)

FINGERPRINT DISPLAY DURING SSH PROMPT

bash-4.2$ ssh -i ./blah.PEM ubuntu@ip-172-31-6-91.us-east-2.compute.internal
The authenticity of host 'ip-172-31-6-91.us-east-2.compute.internal (172.31.6.91)' can't be established.
ECDSA key fingerprint is SHA256:ibwhkrF5oMapJla4cKuXgePT5lHmg08L7yMp6auCpgo.
ECDSA key fingerprint is MD5:ba:82:53:ee:89:22:26:63:26:11:21:93:63:1f:1d:d1.

How could the fingerprints be different, but the key still allows me to connect?

Community
  • 1
Jazzepi
  • 191
  • 1
  • 8
  • 2
    For starters, `known_hosts` doesn't store fingerprints, it stores public keys, so I think you have some more incorrect assumptions to work through. – womble May 14 '19 at 01:06

3 Answers3

7

You have 2 key pairs at play there:

  1. Server's Private/Public key.

ssh daemon on the server has a set of private keys created and stored in the /etc/ssh/ folder

The RSA fingerprint you are getting from the server comes from the public key corresponding to the /etc/ssh/ssh_host_rsa_key private key

  1. User's Private/Public key.

This is a keypair you own. The private key should be securely stored on your computer and used to authenticate to the server. The public key is on the server, in your profile's authorized_keys file: ~/.ssh/authorized_keys

So there are 2 different public keys, and their fingerprints will not match, unless you use the same private key as one on the server, which is unlikely.

To get rid of warning do exactly as it has been asking: put fingerprint of the server into the /var/lib/jenkins/.ssh/known_hosts file.

Sergey Nudnov
  • 863
  • 6
  • 12
  • The process I went through is this. * Use AWS to create a KEY_PAIR. Which gives me the private RSA key for that key pair. AFAIK that's what I'm downloading. I then wanted to store that key in my Jenkins as a method to login to the host with this KEY_PAIR so the jenkins can do some initialization on it through Ansible. Throughout this process I did NOT generate my own key pair, I just have the one I downloaded from AWS when I created the KEY_PAIR instance. – Jazzepi May 14 '19 at 00:42
  • Ohhh I think I understand. When I create a KEY_PAIR in AWS it's giving me the private key, and putting the public key into that host's authorized keys so that I can log in to it. But that means the host generates its own public/private key pair, and I need to scrape the logs to get a handle on that so I can validate the fingerprint. – Jazzepi May 14 '19 at 00:43
  • @Jazzepi, yes, the private key you downloaded is a part of User's key pair. The public key's fingerprint of this key-pair is on the server in your profile – Sergey Nudnov May 14 '19 at 00:45
  • @Jazzepi, usually there is `/etc/ssh/ssh_host_rsa_key.pub`. To get its fingerprint you could use `ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub` – Sergey Nudnov May 14 '19 at 00:49
  • Small nit: `authorized_keys` contains the actual public key, not its fingerprint. – Christoffer Hammarström May 14 '19 at 11:47
  • @ChristofferHammarström Correct. Interestingly it was initially that in the answer, then I corrected it wrongly. Now I corrected it back. :) – Sergey Nudnov May 14 '19 at 11:53
5

If I understand you correctly, the private key file is in your possession and you'd like to get the fingerprint of it so that you can add it into your known_hosts file. If that's right, then here's what you do:

$ ssh-keygen -yf /path_to_private_key/key_file_name

That will output something like:

ssh-rsa AAAAB3NzaC....

Lastly, prefix that with the IP address to which you SSH, so that you have this:

10.200.25.5 ssh-rsa AAAAB3NzaC....

and you can add that as a line in your known_hosts file.

user3629081
  • 325
  • 1
  • 8
  • I put in DNS_ADDRESS ssh-rsa And get the below error complaining about the fingerprints not matching. I deleted some lines. RSA host key for ip-172-31-6-91.us-east-2.compute.internal has changed, and the key for the corresponding IP address 172.31.6.91 is unknown. The fingerprint for the RSA key sent by the remote host is SHA256:9BOOk2Zjlin6CnIyaNFXlLpShcAyQis/wKU8oPCAauE. Add correct host key in /var/lib/jenkins/.ssh/known_hosts to get rid of this message. RSA host key for ip-172-31-6-91.us-east-2.compute.internal has changed and you have requested strict checking. – Jazzepi May 13 '19 at 23:26
  • I'm not sure why they don't match, but it seems that the key you have is not the one the server is using. You can find the key which the server is using in `/etc/ssh/`. In that directory, you will find the private keys and the public fingerprints already generated. – user3629081 May 13 '19 at 23:41
  • The private key PEM file I downloaded from AWS is a RSA key. -----BEGIN RSA PRIVATE KEY----- so I assume that it should match the encoded contents `/etc/ssh/ssh_host_rsa_key` but they don't. I think PEM files can contain comments though? Maybe the comments are throwing it off? – Jazzepi May 13 '19 at 23:57
  • Comments in the PEM file would prevent `ssh-keygen ` from calculating the fingerprint at all. Personally, I would just use the fingerprint found on the server and move on with life. – user3629081 May 14 '19 at 00:11
  • I am honestly just trying to understand why this isn't working. There is definitely a comment embedded in the private key pem file on the server that is not embedded in the one I downloaded from AWS. When I'm on the server I'm trying to connect to I get this root@ comment ubuntu@ip-172-31-6-91:/etc/ssh$ sudo ssh-keygen -lf ./ssh_host_rsa_key 2048 SHA256:9BOOk2Zjlin6CnIyaNFXlLpShcAyQis/wKU8oPCAauE root@ip-172-31-6-91 (RSA) On the client trying to connect I get this, "no comment" 2048 SHA256:TJrXSILH/tgLjqVtuxGBFJH+5HMBCLt0StxLPeS7laY no comment (RSA) – Jazzepi May 14 '19 at 00:36
  • That's because you're comparing RSA to DSA. I think if you add `-o 'HostKeyAlgorithms=ssh-rsa'` to your SSH command, it should work correctly. – user3629081 May 14 '19 at 00:53
  • Thanks for the help. This was so confusing. – Jazzepi May 14 '19 at 00:55
2

My underlying confusion was that I thought I had the exact same pair of private and public keys that the server did. Instead what's happening is when I create a key pair and assign it to a new EC2 instance, the EC2 instance is getting the public key of that pair put into its authorized_keys which allows me to connect to it with the private key that I download when creating the pair in AWS.

I can use the fingerprinting command that comes with AWS, but it's only good to validate that the private key that I have, matches the public key they have stored, and will put into the authorized_keys.

Every time a new EC2 instance comes up, it generates a collection of its own private/public keys for different algorithms like RSA and DSA. I must now scrape the logs to get the fingerprints for those keys so that I can validate that they match the host I'm connecting to.

So the steps are.

  1. Generate the EC2 instance, keep the key you get.
  2. Give that key from step 1 to Jenkins so that it can connect to the host.
  3. Use the get-console-output command to scrape the fingerprints for the keys from the logs.
  4. Attempt to connect to the remote instant with the key from step 1. Use the key fingerprint from that error message to validate against the fingerprint you scraped in step 3.
  5. Once you've validated, then you know it's safe to add the remote host.
  6. Profit!!!

Keep in mind the vital issue here is that you can't trust that the host you're connecting to isn't a man in the middle attack. If you blindly accept the key without validating it's fingerprint in step 4, you may not be connecting to the server you expect to be. By validating in step 4 you know that your connection is secure (because of SSH's cryptography), but crucially you also know WHO you are connected to, because only one person is going to have the key-pair fingerprint matching the one you expect.

EDIT: The get-console-output command is not reliable for automation. It's ONLY intended for ad-hoc troubleshooting. The core problem is that AWS will arbitrarily cut parts of the log out, and/or buffer it in a way that you must wait a long time to see the complete entry.

Instead I'm trying to upload the keys in the user data script, bring the system down, clear the user data script so it's not accessible (because it has a private key in it), and then bring the instance back up. I need to reboot it anyways because updated packages might require a reboot so I can kill two birds with one stone here.

https://alestic.com/2012/04/ec2-ssh-host-key/

Jazzepi
  • 191
  • 1
  • 8