Stack Overflow
Stack Overflow

Questions tagged [apache-tez]

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data.

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data. It is currently built atop Apache Hadoop YARN

See Hive-on-Tez configuration properties.

192 questions
1
vote
1 answer

Hive LLAP throws Unable to process container ports mapping

Im trying to get Hive LLAP to run on my server. My setup so far is: Hadoop 3.31 , tez 0.9.2, hive 3.1.2, zookeper 3.7.0 all from tar files. Hive on Tez is working. Selects return the expected results. Now i wanted to get LLAP running so i setup the…
De Bug
  • 11
  • 1
1
vote
0 answers

Hive job on Tez execution engine fails on memory allocation

org.apache.parquet.hadoop.MemoryManager$1: New Memory allocation 1046531 bytes is smaller than the minimum allocation size of 1048576 bytes.
BengalTiger
  • 11
  • 2
1
vote
0 answers

How to get value of default database from which the tez query was fired via Aplication Timeline Server

I want to get the metadata information about all the TEZ jobs that ran on the hadoop cluster. So I referred to the YARN Timeline Server documentation to get the metadata of jobs done. I have set the…
user8099124
  • 21
  • 3
1
vote
1 answer

java.lang.NullPointerException when merging output files

I have a table with 3 partition columns create table tn( col1 string, etc... ) partitioned by ( time_key date, region string, city string ) stored as orc tblproperties ("orc.compress"="ZLIB"); City partition can have from a few mb to a few hundred…
gjin
  • 860
  • 1
  • 14
  • 28
1
vote
1 answer

% of Queue and % of Cluster difference in yarn ui for MR/TEZ/SPARK jobs run

While debugging the TEZ/SPARK/MR jobs I can see the below jobs details in YARN UI. does any one please help me with % of Queue & % of Cluster stats, since it showing 1815% usage ?
sathya
  • 1,982
  • 1
  • 20
  • 37
1
vote
2 answers

How to pass Hive conf with sqoop job

How to pass hive conf parameter with Sqoop import job. SET hive.tez.container.size=4096 I am using this hive conf from hive cli every time.
A.james
  • 27
  • 4
1
vote
2 answers

hive on tez error:java.lang.OutOfMemoryError

I am facing this error when perform partitioning by date on a hive table that have more 70 columns : ERROR : Status: Failed ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1612203694878_0265_4_00, diagnostics=[Task failed,…
A.james
  • 27
  • 4
1
vote
2 answers

How to Improve Cross Join Performance in Hive TEZ?

I have a hive table with 5 billion records. I want each of these 5 billion records to be joined with a hardcoded 52 records. For achieving this I am doing a cross join like select * from table1 join table 2 ON 1 = 1; This is taking 5 hours to run…
Mani
  • 187
  • 2
  • 5
  • 14
1
vote
1 answer

Persistance of parameter in Hive HQL?

I use a cluster with Hive. The cluster has a specific tez continer size (set via Ambari). However, we have a certain hive operation that processes more data than the other. It is the only one. Consequently, we plan to change tez container size just…
Itération 122442
  • 2,644
  • 2
  • 27
  • 73
1
vote
0 answers

Hive string functions failing with VARCHAR but working with string

I am using jaydebeapi to connect to hive. I want to select the first 3 characters of a VARCHAR(100) using hive. I have columns named col1 and col2. col1 is a string and col2 is a VARCHAR(100) When I do this, it works fine: connection =…
newbieAtLife5741
  • 11
  • 1
1
vote
1 answer

Naming hive session cannot work sometimes

I ran sqls on hive tez by hive -f xxx.sql --hiveconf hive.session.id=sessionName but on the yarn resourcemanager displays like this HIVE-f4ea6c3f-f4cf-4db3-8801-da6f94e20237 HIVE-d920c434-d2e6-4c1c-a506-d69b580960f7 sometimes it displays…
user3065606
  • 235
  • 1
  • 4
  • 13
1
vote
1 answer

Tez View - "Download Data" files location

In Ambari, inside the Tez View, there is a Download Data button at "Details" pane inside each query. If someone press this button, it downloads a ~1MB zipped file called, for example, "dag_1584136991319_12180_1.zip". As I can see, there are 12 json…
Lucas Mignone
  • 55
  • 7
1
vote
3 answers

Apache Tez Job fails due to java.lang.NumberFormatException for input string: "30s"

I am trying to execute query on Apache hive on tez but somehow i am getting the error as below and I have no clue how to solve it Apache Hadoop 3.1.1 Apache Hive 3.1.0 Apache Tez 0.9.1 My tez-site.xml …
arjun jain
  • 11
  • 2
1
vote
1 answer

COUNT(*) returning multiple rows in hive instead one

Why does COUNT(*) return 2 counts in Hive It is observed that summation of both the returned value is equal to actual count of records in the table .Please note that I am not using any where , group by clause in the query. Below is my create…
Surabhi Y
  • 11
  • 2
1
vote
1 answer

error when performing analytic query (e.g select count(*) ) on Apache hive

when i perform simple query on hive like select * from table3 limit 100; , the result is ok. but when i perform select count(*) from table3 ; i encounter with following error : 0: hiv> select count(*) from table3; INFO : Compiling…
Shoobi
  • 88
  • 1
  • 10
1 2 3
…
12 13