0

While connection to kerberized hadoop environment error: [Simba]ImpalaJDBCDriver Unable to connect to server: [Simba]ImpalaJDBCDriver Kerberos Authentication failed.

I've installed cloudera quickstart vm in virtualbox, enabled kerberos, writing java code which connects to imapala db and getting Kerberos Authentication failed error.

public static void main(String[] args) throws Exception {

        Configuration conf = new Configuration();
        conf.set("hadoop.security.authentication", "Kerberos");
        UserGroupInformation.setConfiguration(conf);
        UserGroupInformation ugi = UserGroupInformation
                .loginUserFromKeytabAndReturnUGI("hdfs/quickstart.cloudera@CLOUDERA", "hdfs.keytab");

        Class.forName("com.cloudera.impala.jdbc41.Driver");
        Connection conn = (Connection) ugi.doAs(new PrivilegedExceptionAction<Object>() {
            public Object run() {
                Connection tcon = null;
                try {
                    tcon = DriverManager.getConnection(
                            "jdbc:impala://quickstart.cloudera:21050;AuthMech=1;KrbHostFQDN=quickstart.cloudera;KrbRealm=CLOUDERA;KrbServiceName=hdfs");
                    System.out.println("Connected!");
                } catch (SQLException e) {
                    e.printStackTrace();
                }
                return tcon;
            }
        });

        Statement stmt = conn.createStatement();

        String sql = "show tables";
        System.out.println("Running: " + sql);
        ResultSet res = stmt.executeQuery(sql);
        while (res.next()) {
            System.out.println(res.getString(1));
        }
    }

I have enabled debug mode, exception which I am getting:


    ...
    Client Principal = hdfs/quickstart.cloudera@CLOUDERA
    Server Principal = hdfs/quickstart.cloudera@CLOUDERA
    Session Key = EncryptionKey: keyType=23 keyBytes (hex dump)=
    0000: 8A B3 79 07 A5 06 05 9F   CE 37 84 8A 15 2E 7E B5  ..y......7......


    Forwardable Ticket true
    Forwarded Ticket false
    Proxiable Ticket false
    Proxy Ticket false
    Postdated Ticket false
    Renewable Ticket false
    Initial Ticket false
    Auth Time = Sun Jun 23 11:52:03 PDT 2019
    Start Time = Sun Jun 23 11:52:03 PDT 2019
    End Time = Mon Jun 24 11:52:03 PDT 2019
    Renew Till = null
    Client Addresses  Null 
    >>> KrbApReq: APOptions are 00100000 00000000 00000000 00000000
    >>> EType: sun.security.krb5.internal.crypto.ArcFourHmacEType
    Krb5Context setting mySeqNumber to: 925793988
    Created InitSecContextToken:
    0000: 01 00 6E 82 02 2E 30 82   02 2A A0 03 02 01 05 A1  ..n...0..*......

    0220: 4A 3E 74 0A 67 B6 5E 16   3B B8 1D FB 91 75 53 33  J>t.g.^.;....uS3
    0230: 76 5E 40 81                                        v^@.

    java.sql.SQLException: [Simba][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: [Simba][ImpalaJDBCDriver](500169) Unable to connect to server: [Simba][ImpalaJDBCDriver](500591) Kerberos Authentication failed..
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source)
        at com.cloudera.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
        at com.cloudera.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
        at com.cloudera.jdbc.common.AbstractDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:233)
        at ImpalaJDBC$1.run(ImpalaJDBC.java:64)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
    Caused by: com.cloudera.support.exceptions.GeneralException: [Simba][ImpalaJDBCDriver](500164) Error initialized or created transport for authentication: [Simba][ImpalaJDBCDriver](500169) Unable to connect to server: [Simba][ImpalaJDBCDriver](500591) Kerberos Authentication failed..
        ... 13 more
    Caused by: java.lang.RuntimeException: [Simba][ImpalaJDBCDriver](500169) Unable to connect to server: [Simba][ImpalaJDBCDriver](500591) Kerberos Authentication failed.
        at com.cloudera.hivecommon.api.HiveServerPrivilegedAction.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:356)
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
        at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
        at com.cloudera.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source)
        at com.cloudera.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
        at com.cloudera.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
        at com.cloudera.jdbc.common.AbstractDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:233)
        at ImpalaJDBC$1.run(ImpalaJDBC.java:64)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
        at ImpalaJDBC.main(ImpalaJDBC.java:60)
    Caused by: org.apache.thrift.transport.TTransportException
        at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
        at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:258)
        at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
        ... 17 more

  • What happens when you do "kinit -kt hdfs.keytab hdfs/quickstart.cloudera@CLOUDERA"? Do you get a valid ticket? – facha Jun 23 '19 at 22:43
  • I think yes: [cloudera@quickstart ~]$ kinit -kt hdfs.keytab hdfs/quickstart.cloudera@CLOUDERA [cloudera@quickstart ~]$ klist Ticket cache: FILE:/tmp/krb5cc_501 Default principal: hdfs/quickstart.cloudera@CLOUDERA Valid starting Expires Service principal 06/23/19 16:09:01 06/24/19 16:09:01 krbtgt/CLOUDERA@CLOUDERA renew until 06/30/19 16:09:01 – Ivan Koshelia Jun 23 '19 at 23:12
  • I can see you are doing kinit inside your virtual machine. Are you running your code on the virtual machine as well? I'm asking because one would usually run it elsewhere (e.g. on the host machine inside IDE) – facha Jun 24 '19 at 15:00
  • yes, I am running on virtual machine as well – Ivan Koshelia Jun 24 '19 at 15:25
  • can it be the cause that i didn't enable kerberos for impala, but through jdbc driver i want to connect to impala? I just enabled kerberos on cloudera manager but not for impala – Ivan Koshelia Jun 24 '19 at 15:32
  • Did you run the "enable kerberos" wizard in Cloudera Manager? That should have enabled kerberos auth for all services, including impala – facha Jun 24 '19 at 15:38
  • Try running a more simple example: https://pastebin.com/wY2SXMs7. You will need to get a ticket with kinit before running – facha Jun 24 '19 at 15:59
  • Since you're not setting `java.security.krb5.realm` and `java.security.krb5.kdc` in you program, are you specifying the location of `krb5.conf` file via `java.security.krb5.conf` when calling it? https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html Also +1 to @facha, you need to kerberize your Impala via CM. – mazaneicha Jun 26 '19 at 23:01

1 Answers1

0

After the Impala restart, my web project encountered the same problem. Set the connection pool to release all connections when it is idle, and let the next query re apply for connection. At this time, this problem will not occur.