1

Suppose I had a configuration that included 1) a local Windows client, 2) a remote unix server without windowing capability, and 3) a separate remote hadoop cluster that houses data to be queried with (among other things) Hive. I am seeking a way to establish the Hive Metastore as a data source in a Jetbrains IDE installed on the Windows client (specifically Intellij).

The wrinkle in this configuration is Kerberos, which is installed on the remote unix server, but not on the local Windows machine. Typically, the Hive Metastore is accessed from the unix server. It should be assumed that installing Kerberos on the Windows client is not a feasible scenario, and it isn't clear to me how Intellij could feasibly be used on a windowless unix environment in any scenario. However, I really want the features it provides to be available.

  1. Is it actually possible to get Intellij to somehow leverage the ability to initialize a Kerberos ticket on the unix server to connect to Hive?

  2. Is it possible to get Intellij to reactively query my Kerberos credentials upon initialization of a connection with the Hive Metastore?

This seems less than likely, but any ideas would be greatly appreciated.

Marvin Ward Jr
  • 1,019
  • 1
  • 11
  • 30
  • _"establish the Hive Metastore as a data source"_ > how do you define "data source"?? The Metastore is a Java service (backed by a RDBMS e.g. Derby or MySQL) accessible via a Java API or via Thrift wire protocol. – Samson Scharfrichter Apr 12 '18 at 07:41
  • _"Kerberos ... installed ... not on the local Windows machine."_ > no kidding?! Kerberos is the pillar of Active Directory authentication, via SSPI proprietary extensions. And Java supports Kerberos out of the box (in a crude way) via GSSAPI -- just check out the `klist.exe` in Windows system dir vs. in Java install dir – Samson Scharfrichter Apr 12 '18 at 07:46
  • You can even SSO from a Java app on Windows to a Hive JDBC server (or a Metastore server via API) if they are backed by Active Directory as KDC. Though it requires some tweaking of the JAAS configuration. – Samson Scharfrichter Apr 12 '18 at 07:49
  • Thanks for responding Samson. I do realize that the MetaStore is a service, but the Hive MetaStore is a data source in this case from the perspective of Intellij. I am not seeking to connect with a standalone Java app, I need Intellij to connect via it's own connection infrastructure so that it is more readily deployed by others. (I have via JDBC, but am unclear on auth.) As for the native Java support for Kerberos, I didn't know that. Would you mind elaborating on the approach you would use in an answer (potentially expanding the shorthand you have used)? I'd be happy to give it a go. – Marvin Ward Jr Apr 12 '18 at 10:29
  • Cf. https://stackoverflow.com/questions/42477466/error-when-connect-to-impala-with-jdbc-under-kerberos-authrication/42506620 for sample JAAS conf (using keytab file for password) & tweaks -- the Metastore expect the same codename as Apache Hive JDBC driver since they share the same code base – Samson Scharfrichter Apr 12 '18 at 18:31
  • For a popup prompting for password instead of keytab, it's a bit more complex, you have to unlock the feature from JVM `security` policy -- or extend it via a system property (...) – Samson Scharfrichter Apr 12 '18 at 18:37
  • For more context on JAAS https://stackoverflow.com/questions/45770743/kerberos-error-connecting-to-impala-and-hbase – Samson Scharfrichter Apr 12 '18 at 18:37
  • For more context about Kerberos (and Hadoop) https://stackoverflow.com/questions/33211134/hbase-kerberos-connection-renewal-strategy – Samson Scharfrichter Apr 12 '18 at 18:40
  • For pointers about Windows extra reqs, cf. comments on https://stackoverflow.com/questions/47533532/hivemetastoreclient-fails-to-connect-to-a-kerberized-cluster – Samson Scharfrichter Apr 12 '18 at 18:44
  • I'll give this a go as soon as I can, but in the meantime, it did look like the last link suggested a need to install an executable on Windows to enable tickets. Am I reading that correctly? I can't really install anything new on any of these machines. Is that a show stopper without significant development time? – Marvin Ward Jr Apr 12 '18 at 19:51
  • _MIT Kerberos for Windows_ has a UI, stores its cache in memory, is supported by Firefox and Chrome (for SSO) and also by Java. But Java does not require it -- the sample JAAS conf in 1st link creates a ticket on-the-fly, and there's a command line to create tickets in a file cache (the Linux way) https://docs.oracle.com/javase/8/docs/technotes/tools/windows/kinit.html – Samson Scharfrichter Apr 13 '18 at 06:43

0 Answers0