Stack Overflow
Stack Overflow

Questions tagged [tez]

The Apache Tez™ project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data. It is currently built atop Apache Hadoop YARN.

The 2 main design themes for Tez are:

Empowering end users by: Expressive dataflow definition APIs Flexible Input-Processor-Output runtime model Data type agnostic Simplifying deployment Execution Performance Performance gains over Map Reduce Optimal resource management Plan reconfiguration at runtime Dynamic physical data flow decisions

[For more details visit this link][1]: https://tez.apache.org/

94 questions
0
votes
1 answer

Hive Testbench data generation failed

I cloned the Hive Testbench to try to run Hive benchmark on a hadoop cluster built with Apache binary distributions of Hadoop v2.9.0, Hive 2.3.0 and Tez 0.9.0. I managed to finish the build of the two data generators: TPC-H and TPC-DS. Then the…
robert
  • 397
  • 1
  • 3
  • 14
0
votes
1 answer

Task each Mapper Performs

I have a sql which has multiple joins. 2017-12-28 05:15:53,140 Map 1: 15/15 Map 6: 9/9 Map 7: 29/29 Map 8: 198(+2)/200 Map 9: 1/1 Reducer 2: 1/1 Reducer 3: 0(+1)/1 Reducer 4: 0(+3)/3 Reducer 5: 0(+11)/11 Above is the line from the log…
user1206412
  • 63
  • 1
  • 10
0
votes
1 answer

Tez vertex error shown while execution of query in hive

can any one explain what is VERTEX_FAILURE error in Hive while using Tez execution engine ? Also what is the root cause of it?
Manish Pansari
  • 371
  • 4
  • 9
0
votes
1 answer

Impala 2.7 fails to read any data from a parquet table created from Hive with Tez

I'm populating a partitioned Hive table in parquet storage format using a query that is using a number of union all operators. Query is executed using Tez, which with default settings results in multiple concurrent Tez writers creating HDFS…
sokhaty
  • 418
  • 3
  • 8
0
votes
1 answer

Tez - DAGAppMaster - java.lang.IllegalArgumentException: Invalid ContainerId

I try to launch a mapreduce job, but I get an error while excuting the jobs in shell or in hive : hive> select count(*) from employee ; Query ID = mapr_20171107135114_a574713d-7d69-45e1-aa73-d4de07a3059b Total jobs = 1 Launching Job 1 out of 1…
Ayman Anikad
  • 91
  • 8
0
votes
1 answer

what's the purpose of spark sql over hive?

I know the question had been asked years ago, but I am still wondering the true purpose of using SparkSQL / HiveContext. Spark approach gives a more generic distributed way that the built-in MapReduce. I read a lot of articles claiming that MR way…
Sebastien
  • 115
  • 1
  • 11
0
votes
1 answer

Add Filesystem to Hive with Tez execution engine

I receive the following error when executing a query with the Tez execution engine against an external ORC table: Vertex failed, vertexName=Map 6, vertexId=vertex_1504790331090_0008_1_01, diagnostics=[Vertex vertex_1504790331090_0008_1_01 [Map 6]…
jscott
  • 397
  • 2
  • 5
  • 18
0
votes
1 answer

How can I add additional libraries to Tez to support a different HDFS backend?

When I use MapReduce as the execution engine behind Hive, I am able to use an alternate backend to my defaultFS.impl. Using syntax similar to: LOCATION 'protocol://address:port/dir'; I would like to use the Tez execution engine instead of MapReduce,…
jscott
  • 397
  • 2
  • 5
  • 18
0
votes
0 answers

hive on tez, cannot monitor timeline in Tez UI

I configured hive on tez as following. I can see my queries in HiveServer2 Web UI. But, I cannot see DAGs in Tez UI after I submit hive queries. What configurations am I…
planaria
  • 47
  • 7
0
votes
0 answers

Pig script running on MapReduce but not on Tez

I am using version of Pig(0.16.0) and Tez version is 0.9.0. The pig script is running fine on MapReduce, but not with Tez. I had tried change tez-0.8.(3-5) still not work. Can this be a version mismatch problem? Please have a look at the logs: ERROR…
Perling
  • 3
  • 1
  • 5
0
votes
1 answer

Can we impose custom DAG plan over generated DAG plan in TEZ? If possible, How?

Instead of using a DAG plan generated by TEZ in Hive, I want to use custom DAG plan created by me. Is it Possible? If Yes, can anyone explain?
Ramprakash
  • 151
  • 1
  • 8
0
votes
1 answer

Unrecognized SSL message, plaintext connection node, while building apache tez

While building apache tez using maven, got the error. Error: Unrecognized SSL message, plaintext connection node Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm
Nagendra
  • 51
  • 4
0
votes
2 answers

Error when using "Collect_List()" function in hive

Whenever I run function "collect_list" on Hive, it always throws an error: Query ID = xxxxx Total jobs = 1 Launching Job 1 out of 1 Failed to get session FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask Here's…
Deo
  • 1
  • 2
0
votes
1 answer

HIVE on TEZ (java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support)

My script keeps falling with the same error java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support The code itself looks like this WITH step1 AS( SELECT columns FROM t1 stg …
gjin
  • 860
  • 1
  • 14
  • 28
0
votes
1 answer

hive on tez is not supporting hdfs federation

hive version is apache hive-2.1.0 tez version is apache tez-0.8.4 i set tez.lib.uris=viewfs://nsX/nameservice/hive/lib/tez-0.8.4-zdh7.1.1-SNAPSHOT-minimal.tar.gz when i run a tez job on hive ,it always appears the problem 2017-03-23T15:26:24,255…
user7743117
  • 1
1 2 3 4
5
6 7