Stack Overflow
Stack Overflow

Questions tagged [tez]

The Apache Tez™ project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data. It is currently built atop Apache Hadoop YARN.

The 2 main design themes for Tez are:

Empowering end users by: Expressive dataflow definition APIs Flexible Input-Processor-Output runtime model Data type agnostic Simplifying deployment Execution Performance Performance gains over Map Reduce Optimal resource management Plan reconfiguration at runtime Dynamic physical data flow decisions

[For more details visit this link][1]: https://tez.apache.org/

94 questions
1
vote
2 answers

"INFO : Tez session hasn't been created yet. Opening session" hang

I'm working with HDP 2.3.0.0-2557 and I'm trying to run a hive query like the following: beeline> !connect jdbc:hive2://localhost:10000 Connecting to jdbc:hive2://localhost:10000 Enter username for jdbc:hive2://localhost:10000: root Enter password…
xxks-kkk
  • 2,336
  • 3
  • 28
  • 48
1
vote
3 answers

Apache Tez build failed because of tez-ui

I cannot build Apache Tez with tez-release-0.7.0 because of failure on tez-ui. This is my command in Ubuntu: mvn clean package -DskipTests=true -Dmaven.javadoc.skip=true -Dfrontend-maven-plugin.version=0.0.23 Where should I fix it now? Thank you…
hminle
  • 501
  • 1
  • 8
  • 19
1
vote
0 answers

Tez error-INIT_FAILURE, Fail to create InputInitializerManager

I have installed apache-tez-0.8.1-alpha ,hadoop 2.7.2 and hive-2.0.1 in my machine.I changed mapreduce framework as "yarn-tez".so i can run hadoop applications using Tez.But when i run apllications using hive queries by setting…
vasanth
  • 224
  • 3
  • 19
1
vote
0 answers

hive job failed on java.io.FileNotFoundException

hive> select distinct symbol from minute_bars; WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Query ID =…
PrayerHuang
  • 31
  • 4
1
vote
0 answers

Hadoop cluster. Large number of TCP retransmissions

We have hadoop2 cluster including 2 namenodes, 13 datanodes and 1 'mediator' machine with DB for meta. Namenodes and datanodes are in the same data center, but meta DB is on remote side. When executing MR jobs there are a lot of TCP retransmissions…
Samriang
  • 403
  • 6
  • 18
1
vote
2 answers

Record too large for in-memory buffer. Error when working with Hive's ORC tables via TEZ

We are trying to read data from 'ORC' table in HIVE (1.2.1) and put that data into table with 'TextInputFormat'. Some entries are too large in original data and following error occurs during…
Samriang
  • 403
  • 6
  • 18
0
votes
0 answers

Preemption with Tez along with the yarn FairShare scheduler supported?

We've been switching our 10 nodes cluster from MapReduce to Tez lately and we are experiencing issues with resource management since then. It seems like preemption does not work as expected : a very consuming job arrives it gets all free…
mazelx
  • 142
  • 8
0
votes
0 answers

Hive’s dynamic partitioning failing to write final files

I’m trying to load data from a table with 1 column partitioned to a new table that has 2 partitioned columns, with the newer partitioned column being a regular column from the first table. For example the create table statements (simplified and…
dl8
  • 1,270
  • 1
  • 14
  • 34
0
votes
1 answer

Hive complex data type querying

I am trying to query data from source table but I am unable to see the result properly. Source table structure c1 string, c2 string, c3 string, temp1 struct < s1 : string, s2 : string, s3 : string, temp2 : array> …
akm
  • 113
  • 1
  • 2
  • 10
0
votes
1 answer

How many number of reducers will be running if I use distinct on all columns in hive?

I am running a hive query where distinct is applied on all the selected columns, and I noticed that there are a couple of reducers being initiated for running the same query. can anyone explain me the reason behind it? Example query: > SELECT…
A srinivas
  • 791
  • 2
  • 9
  • 25
0
votes
1 answer

hive llap - which execution engine supported? spark,mr, tez

The documentation is not clear. All blog post examples about Hive LLAP (Long Live and Process) uses the Tez execution engine, but can Spark/MR hive engines also use LLAP?
tooptoop4
  • 234
  • 3
  • 15
  • 45
0
votes
1 answer

Hadoop Hive TEZ task fails (to start container)

Ive got hadoop-2.8.3, hive-2.3.2, and tez (trying out 0.9 and 0.8.5 versions). tez-site basic conf: tez.lib.uris: /apps/tez/apache-tez-0.8.5-bin.tar.gz (uploaded to hdfs) tez.use.cluster.hadoop-libs: false Container log stderr shows: Error: Could…
strzelecki.maciek
  • 324
  • 2
  • 22
0
votes
1 answer

Pig script on aws emr with tez occasionally fails with OutOfMemoryException

I have a pig script running on an emr cluster (emr-5.4.0) using a custom UDF. The UDF is used to lookup some dimensional data for which it imports a (somewhat) large amout of text data. In the pig script, the UDF is used as follows: DEFINE…
huch
  • 675
  • 8
  • 13
0
votes
2 answers

OOM in tez/hive

[After a few answers and comments I asked a new question based on the knowledge gained here: Out of memory in Hive/tez with LATERAL VIEW json_tuple ] One of my query consistently fails with the error: ERROR : Status: Failed ERROR : Vertex failed,…
Guillaume
  • 2,325
  • 2
  • 22
  • 40
0
votes
2 answers

Getting a Intermittent issue while running Delete or Update in Hive

I have a very strange behavior with my cluster, I am running Delete/Update statement in Hive cli which sometime works fine and some time fails below is the Command. Table is ACID enabled. hive> delete from temptable where name='Jose'; Exception is…
Anupam Alok
  • 433
  • 3
  • 17
1 2 3
4
5 6 7