Stack Overflow
Stack Overflow

Questions tagged [apache-tez]

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data.

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data. It is currently built atop Apache Hadoop YARN

See Hive-on-Tez configuration properties.

192 questions
1
vote
1 answer

Reducing mappers and reducers for a simple query on a very large table/view in Hive

We have a simple query running on a table/view which is approx of size 5 TB. We are performing ETL and finally adding the data to the core table by adding a partition. But as the data we are processing is huge, the query spawns 4000+ mappers and…
akash sharma
  • 411
  • 2
  • 24
1
vote
0 answers

Need to capture hive query from application id

I want to find the hive query from application id, I know that in TEZ view I can see the query, but i want to know if there is any api which I can use to find query from application id from command line or through curl command?
Nick
  • 11
  • 2
1
vote
2 answers

How do I calculate the most popular device, os, browser in each city in HiveQL?

I have a table with user agent strings (which I parse into browser, os, and device columns) and city id's. I want to calculate the most popular browser, os and device for each city. Here's my attempt: select device os, browser, name, MAX(hits) as…
pavel_orekhov
  • 1,657
  • 2
  • 15
  • 37
1
vote
1 answer

What hive property to set to avoid cross product?

I am running hive (Version 1.2) query on Tez and my query is been taking forever to return data because of the cross product . I have already tried some of these hive properties. set hive.execution.engine=tez; --set hive.execution.engine=mr; set…
cheapcoder
  • 183
  • 1
  • 3
  • 12
1
vote
0 answers

Apache slider LLAP container fails to start

I am trying to launch an LLAP container and I see the following error in the container log Log Type: slider-agent.out Log Upload Time: Fri Nov 30 02:01:10 -0800 2018 Log Length: 783 Traceback (most recent call last): File…
Vijay Muvva
  • 1,063
  • 1
  • 17
  • 31
1
vote
1 answer

Hive TEZ is taking very long time to run the query

I'm kinda of new to Hive and Hadoop . I have a query which is taking 10 minutes to complete the query . Size of the data is 10GB Statistics:Num rows: 4457541 Data size: 1854337449 Basic stats: COMPLETE Column stats: COMPLETE Partition and…
Varshini
  • 69
  • 10
1
vote
0 answers

Sqoop and Hive Tez in a Ec2 instance are not running properly

I have the following set in my mapred-site.xml in my ec2 inctance where my Hadoop is working : mapreduce.map.memory.mb 2048 mapreduce.reduce.memory.mb …
Vijaya Seetharaman
  • 161
  • 4
  • 13
1
vote
1 answer

Failure while running hive query - Map operator initialization failed - OOM

I am executing hive query which is created by joining 3 tables.But I am geting below error i.e.Please let me know what this error is and how can I resolve this error. org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1,…
cheers519
  • 445
  • 6
  • 18
1
vote
1 answer

why mappers and reducers are allocated on hive tez?

Can anyone please explain even when running hive query in TEZ, yet Mappers and Reducers are allocated? Is running in Tez engine still uses MR engine?
SCode
  • 44
  • 7
1
vote
0 answers

Hive Error- while copying data from one DB table to another DB table

I want to copy data from one DB table to another DB table using hive on EMR. Below is the HQL using which I'm copying data along with the date partition. insert into Target.exttbl_user_identification_details PARTITION(load_date="2018-04-23") select…
Ganesh
  • 61
  • 10
1
vote
1 answer

vertext failled Error and Mapper initialized failed - Hive

I'm using Hortonworks data platform in our server with 2 nodes. I'm running query successfully in hive. Suddenly I'm facing mapping with source table to add column to my new table, By this below query; while running this query in hive view. How do I…
Varun_Rathinam
  • 274
  • 1
  • 5
  • 19
1
vote
1 answer

Why am I getting negative allocated mappers in Tez job? Vertex failure?

I'm trying to use the PhoenixStorageHandler as documented here, and populate it with the following query in beeline shell: insert into table pheonix_table select * from hive_table; I get the following breakdown of the mappers in the Tez…
TheElysian
  • 159
  • 1
  • 1
  • 7
1
vote
1 answer

HIve2 with Tez gives Execution Error

I am using Hive2 with Tez. When I run the query it gives execution error which shown below. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask ERROR [432a4475-d246-4596-ad4c-54de6fea86c8 main] exec.Task: Failed…
Sahil Desai
  • 3,418
  • 4
  • 20
  • 41
1
vote
0 answers

Hive on tez engine

Currently in our production environment we use hive on tez instead of mapreduce engine ,so i wanted to ask will all the hive optimization for joins will be relevant for tez as well? for example in multitable table it was mentioned that if join key…
Abid
  • 140
  • 1
  • 11
1
vote
1 answer

Failed to create InputInitializerManager error - TEZ on HIVE

I have installed Apache Tez 0.8.1, Hadoop version 2.7.0 and Hive version 2.01.I am able to successfully run the Map Reduce Jobs.But when I configure hive and tried to run a simple count query, it returned the below error.From the error it is trying…
Avinash
  • 127
  • 2
  • 13
1 2 3
…
12 13