0

I already had a Hadoop 3.0.0 cluster consisting of 2 machine: 1 namenode + RM and 1 datanode. I tried to install Apache Hive 3.0.0 by following this document.

When I run schematool -dbType derby -initSchema --verbose on Cygwin, an exception was thrown:

$ schematool -dbType derby -initSchema --verbose
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/BigSol/apache-hive-3.0.0-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/BigSol/hadoop-3.0.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :    org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:       APP
Starting metastore schema initialization to 3.0.0
org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for initialization: 3.0.0
org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified for initialization: 3.0.0
        at org.apache.hadoop.hive.metastore.MetaStoreSchemaInfo.generateInitFileName(MetaStoreSchemaInfo.java:137)
        at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:580)
        at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:562)
        at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1445)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
*** schemaTool failed ***

When viewing the line of code that thrown the exception, I found that Hive tried to find a SQL schema located at %HIVE_HOME%\scripts\metastore\upgrade\derby\hive-schema-3.0.0.derby.sql.

I doubt that Cygwin messed up the path so that Hive didn't find that schema.

My questions:

  1. How can I correct the path (or fix the problem)?
  2. Are there batch files equivalent to *.sh files in %HIVE_HOME%\bin directory as Hive 2.1.1 have?
Cœur
  • 37,241
  • 25
  • 195
  • 267
phqb
  • 351
  • 3
  • 14
  • `%HIVE_HOME%` is a variable windows style. Probably it was supposed to be replaced during installation or run by CMD and not through bash. I guess the value should be `C:\BigSol\hadoop-3.0.0` – matzeri Oct 09 '18 at 19:46
  • I converted all env variables to linux style. I checked by `echo $HIVE_HOME` and it printed `cygdrive/c/BigSol/apache-hive-3.0.0-bin`. I don't think `$HIVE_HOME` is hadoop path because in the getting started guide there is a command `export PATH=$HIVE_HOME/bin:$PATH` adding hive binaries to path. – phqb Oct 10 '18 at 02:21
  • @phqb I have encountered the same problem. Could you please share workaround for the same. – Jon Andrews Jun 04 '20 at 11:49

1 Answers1

0

I found the solution. After running schematool on a Linux machine and copied metastore_db directory to Windows machine, I managed to start HiveServer2 but the beeline CLI said that the jar in C:\cygdrive\c\BigSol\apache-hive-3.0.0-bin\lib\hive-beeline-3.1.0.jar was not found.

It turned out that java in Cygwin parse the wrong path. I made a symbolic link from C:\cygdrive\c to C:\ and it worked.

phqb
  • 351
  • 3
  • 14
  • as `java` is not a cygwin program it does not understand the cygwin `/cygdrive` path. Your workaround is original. – matzeri Oct 10 '18 at 06:21