"Pull" Kerberos / Freeipa Keytabs from IPA server -> to local server

Question

I am in a rather unenviable position of Administering a Hortonworks (Ambari) installation, which is using Kerberos, in a completely closed system.

Here is the problem:

I have found an inconsistency in the keytab file (let's call it foo) on server x:

$klist -kte /etc/security/keytabs/foo.keytab

KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
12 10/21/15 16:24:27 foo@myrealm (xxxxxxxxxxxxx)
12 10/21/15 16:24:27 foo@myrealm (xxxxxxxxxxxxx)

On the free-ipa server, I run the following command:

$ kadmin.local -q 'get_principal foo'
Authenticating as foo with password.
Principal: 
Expiration date: 
Last password change: 
Password expiration date: [none]
Maximum ticket life: 
Maximum renewable life: 
Last modified: xxxxx (foo)
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 4
Key: vno 21, abs-my-security-type, xxxxxx
Key: xxxxxxx
Key: xxxxxxx
Key: xxxxxxx
[snip]
Attributes: REQUIRES_PRE_AUTH
Policy: [none]

The versions "for foo" are different. Version 21 on the freeipa server. Version 12 on the keytab.

I have discovered the "ipa-getkeytab", which will get me a keytab, BUT increments the version number (on the IPA server), whenever I run it.

My version of ipa-getkeytab does not the -r, --retrieve switch (which preserves the version).

I need to get this keytab on 7 servers (Hadoop nodes). I cannot use scp, ssh, as closed network.

Is there a way for me create (pull the foo keytab) locally without incrementing the version number?

:0/

Thanks

Miles.

Answer 1

If your version of ipa-getkeytab does not support -r option for retrieval of keytabs, then your only other option is to retrieve keytabs at one server once and transfer them securely to the other servers.

Sorry, no other options. FreeIPA implementation of KDC database driver (DAL driver) does not allow to assign specific admin rights for remote kadmin connections, thus you cannot even execute 'getprinc' or 'listprincs' commands.

Answer 2

Troubleshooting Kerberos keytabs

A keytab is a file containing pairs of Kerberos principals and encrypted keys (which are derived from the Kerberos password). You can use a keytab file to authenticate to various remote systems using Kerberos without entering a password.

Keytab files are commonly used to allow scripts to automatically authenticate using Kerberos, without requiring human interaction or access to password stored in a plain-text file.

Kerberos keytabs contain "version numbers"

The version stored in the keytab file must be the same as inside Kerberos. The ERROR (in the question) is caused by this situation.

You need to be able to troubleshoot the keytab files, and Kerberos.

Keytab files - (in Hortonworks / Ambari) these are stored in the /etc/security/keytabs directory:

[root@box keytabs]# ls -ltr
total 24

-r--r-----. 1 hdfs      hadoop    1054 Oct 26 14:24 foo.keytab

A practical example of "fixing" the foo.keytab.

Use the klist command to show the principal, and version (KVNO) stored:

$ sudo su -
[root@box ~]# cd /etc/security/keytabs/
[root@box keytabs]# klist -kte foo.keytab
Keytab name: FILE:hdfs.headless.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
  27 10/26/15 14:08:16 foo@principal(******)
  27 10/26/15 14:08:17 foo@principal(******)
  27 10/26/15 14:08:17 foo@principal (******)
  27 10/26/15 14:08:17 foo@principal(******)

The principal stored in the foo.keytab has a version number of 27.

Comparing the version numbers in Kerberos

Use the kadmin.local command to query Kerberos principles (on your Kerberos / freeipa box):

[root@kerbbox ~]# kadmin.local
Authenticating as principal admin/admin@***** with password.
kadmin.local:  getprinc foo@principal
Principal: foo@principal
Expiration date: [never]
Last password change: Mon Oct 26 14:08:17 GMT 2015
Password expiration date: Tue Jan 19 14:19:33 GMT 2016
Maximum ticket life: 1 day 00:00:00
Maximum renewable life: 7 days 00:00:00
Last modified: Mon Oct 26 14:08:17 GMT 2015 (admin/admin@*********)
Last successful authentication: Mon Oct 26 14:36:33 GMT 2015
Last failed authentication: Mon Oct 26 12:33:14 GMT 2015
Failed password attempts: 0
Number of keys: 4
Key: vno 34, ******, *****
Key: vno 34, ******, *****
Key: vno 34, ******, *****
Key: vno 34, ******, *****
MKey: vno 1
Attributes: REQUIRES_PRE_AUTH
Policy: [none]
kadmin.local:
kadmin.local:

Use the getprinc command to get detail for:

• foo@principal (version 34)

  Check the version numbers (in the keytab file, and Kerberos), ensuring they are the same.

Recreating a keytab file

If the version numbers are different in your keytab file (from in Kerberos). You will need to recreate (the foo.keytab).

To check what principles are in a keytab, use the klist -kte keytab file command

• Create a foo.keytab file by using the kadmin.local, and the ktadd command:

  [root@kerbbox ~]# kadmin.local
  
  ktadd -k /root/foo.keytab foo@principal
  

  Sample output (version 11 for the foo@principal):

  11 10/09/15 14:41:58 foo@principal (*******)
  11 10/09/15 14:41:58 foo@principal (*******)
  11 10/09/15 14:41:58 foo@principal (*******)
  11 10/09/15 14:41:58 foo@principal (*******)
  

  Every time you run the ktadd command, the version number (of the principle) will be incremented.

  Copy the foo.keytab to the servers in your cluster, and make sure permissions / ownership are retained:

  # chmod 440 foo.keytab
  # chown hdfs:hadoop foo.keytab
  
  # klist -kte hdfs.headless.keytab
  
  # ls -ltr
  -r--r-----. 1 hdfs      hadoop    1054 Oct 26 14:24 foo.keytab
  

  The version number in the keytab, and kerberos need to be the same.

  Restart the Ambari Service

  You will see a "green" Ambari web-ui.