Authenticating to Kerberos enabled Hadoop Cluster Using Java

Question

I have Kerberos and have enabled Hadoop cluster. I need to perform HDFS operations using Java code.

I have keytab file and username.

Can someone please suggest how can I authenticate Kerberos using Java code?

Thanks.

Answer 1

If you just want to use the HDFS Java API for a short-lived connection, then UserGroupInformation has a loginUserFromKeytab method.

If you want a durable connection (i.e. longer than the lifetime of the initial Kerberos TGT) then have a look at that post.

If you are interested in the dark side of Kerberos (well, there is no other side anyway) then have a look at the "Madness beyond the Gate" online book -- at your own risk.

~~~~~~

BTW there is an alternative to the Java API: the WebHDFS REST service. But on Windows you still need some Java code to create the Kerberos TGT (using GSSAPI i.e. the Linux way, not SSPI i.e. the Microsoft way!!!), run the SPNEGO initial authentication, and retrieve the delegation token. At this point any HTTPS library will be able to handle the file operations using that token.