0

I have a one problem in hive.

When a hive query is executed, large tables can not be generated application

Not appears app on yarn monitoring web page. and beeline is still ready

But the small size table works normally.

I do not even know why an app can not be registered.

As I Know, command the query in the hive, the app is registered.(even if resources are insufficient)

the table size is 3.3TB(parquet format)

and my cluster environment is looks like this

Hadoop version : CDH5.8(hadoop 2.6)

Hive : 1.1

node num : 17 datanode

memory total : 1.6TB

core total : 136

hive execution engine : I tested both mr and spark. But same result.

Lee. YunSu
  • 416
  • 6
  • 21
  • What do you see in YARN? Accepted state? – OneCricketeer Mar 20 '17 at 01:50
  • No application in any state are registered. – Lee. YunSu Mar 20 '17 at 02:07
  • 1
    how many data files in this large table? it's takes some time when there are too many files . may be there is a OOM exception after a while because of too many HDFS object created . – Mike Gan Mar 20 '17 at 10:10
  • Yes my table has a lots of file because of partitionning. I create a non partition table and stored a same data. then I find that query is running. Thank you very much!!! – Lee. YunSu Mar 22 '17 at 04:24

0 Answers0