0

I'm trying to build an Apache Kylin cube, but failed at step 3 with following log msg:

java.lang.RuntimeException: java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf table not found) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:110) at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:119) at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:103) at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf table not found) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:105) ... 11 more Caused by: NoSuchObjectException(message:default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf table not found) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_core(HiveMetaStore.java:1946) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1899) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99) at com.sun.proxy.$Proxy47.get_table(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1332) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150) at com.sun.proxy.$Proxy48.getTable(Unknown Source) at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:180) at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105) at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88) at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95) ... 13 more result code:2

The message shows kylin can't find table "default.kylin_intermediate_t3cube_47ec8760_55ce_445d_a91b_0c33957690bf ", but when i login into hive, the very table exists.

would anyone can give me some light to fix this?

EDIT: I've fix this problem. Kylin use HCatelog to fetch table info but I did not config hive properly.

chenju
  • 69
  • 4

1 Answers1

1

Kylin connects to hive metastore directly via hcatalog lib. That requires hive-site.xml on classpath and it must contain sufficient information (like 'hive.metastore.uris') to connect to the correct hive metastore. Otherwise, if connected to the wrong metastore, such no-table-found exception will occur.

Check the MR configuration of the failed "Extract Fact Table Distinct Columns" job in YARN resource manager or job history server. Search for classpath properties like 'mapreduce.application.classpath'. Check hive-site.xml is on classpath and it contains correct metastore connectivity information.

Note there is "$KYLIN_HOME/conf/kylin_job_conf.xml" that you could use to override any MR job configuration, such like 'mapreduce.application.classpath'.

KYLIN-2530 is an improvement fix. Suppose this issue will be eased start from Kylin 2.0.

Li Yang
  • 284
  • 1
  • 6