Stack Overflow
Stack Overflow

Questions tagged [apache-tez]

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data.

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data. It is currently built atop Apache Hadoop YARN

See Hive-on-Tez configuration properties.

192 questions
2
votes
1 answer

Hive on TEZ query taking forever at Reducer cross product

I have 2 tables: db1.main_table (32 GB) db2.lookup_table (2.5 KB) lookup table has just one column named id, which is also present and is the primary key of main_ table. The goal is to just look at the values in lookup table and delete from…
akki
  • 71
  • 7
2
votes
2 answers

Hive queries failing on Tez but succeeding on Map-Reduce when connecting from Beeline

I am running into a weird error. I am running a simple select * query with a where clause, following is the summary of query execution status Connecting to Hive from EMR (Tez engine) - succeeding Connecting to Hive from EMR (MR engine) -…
Kshitij Kohli
  • 4,055
  • 4
  • 19
  • 27
2
votes
1 answer

Confirmation on re-written query

Original query: SELECT CAST(cust_mart.acct_identifier AS STRING) as f0 FROM cts_work.cust_xref cust_mart GROUP BY cust_mart.f0; Can I replace the above query with below query : SELECT DISTINCT CAST(cust_mart.acct_identifier AS STRING) as…
Vidya K
  • 103
  • 8
2
votes
2 answers

hive application shows running even after killing from command line

I ran a hive query on a decently large dataset and it was taking too much time for the query so I decided to kill the application with : yarn kill -application-id Now when I check from the CLI with: yarn application -list then the above…
poodah nray
  • 21
  • 5
2
votes
1 answer

spark-sql : Error in session initiation NoClassDefFoundError: org/apache/tez/dag/api/SessionNotRunning

I am facing issue while initiating spark-sql session. Initially when I initiated spark session only default database was visible (Not default database of Hive but same of Spark). In order to view hive databases I copied hive-site.xml from…
Shantanu Sharma
  • 3,661
  • 1
  • 18
  • 39
2
votes
1 answer

Diagnostics:Application killed by user

Any idea why my application gets killed? I am the only user of this nodes. Also my history server doesn't show anything. How I trace this down further and realize who killed it and why and how to fix…
Mona Jalal
  • 34,860
  • 64
  • 239
  • 408
2
votes
3 answers

Hive tez execution error

I am running a hive query and I got the following error when setting the hive.execution.engine=tez, while the query is working under engine=MR. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask My query is an…
Vicky
  • 1,465
  • 2
  • 12
  • 21
2
votes
0 answers

Error in running tez with pig from hue.Is it a config issue or I'm using wrong argument

I was doing some exploration with pig+tez and found that it works in CLI with the inclusion of an argument -x tez with pig. However while running it from hue the same should work with a pig argument -exectype tez ,but it failed with following…
0xF
  • 546
  • 4
  • 20
1
vote
0 answers

Unknown frame descriptor while reading zst file on hive?

I have a file a.csv.zst and i ve created textfile table. When i select on hive i see first 100 rows without an error but when i select count(*) it gives an error unknown frame descriptor I can decompress my file on linux with zstd -dc a.csv.zstd How…
CompEng
  • 7,161
  • 16
  • 68
  • 122
1
vote
1 answer

"No enum constant org.apache.orc.CompressionKind.ZSTD" When Insert Data to ORC Compress ZSTD Table

I have created a table in hive 3.1.3 as below; Create external table test_tez_orc_zstd ( Id bigint )stored as orc Tblproperties(orc.compress=zstd) Location '...' It is created, and then I wanted to insert one row; Insert into…
CompEng
  • 7,161
  • 16
  • 68
  • 122
1
vote
1 answer

Hive INSERT OVERWRITE creates 1 large file in every partition

I am executing a hive INSERT OVERWRITE query. SET hive.merge.mapredfiles=false; SET mapred.reduce.tasks=${hiveconf:NUM_REDUCERS}; SET hive.mapred.supports.subdirectories=TRUE; SET mapred.input.dir.recursive=TRUE; SET hive.execution.engine=tez; SET…
Ishan Tiwary
  • 938
  • 6
  • 15
  • 37
1
vote
1 answer

Invalid Data in Hive-Created Table?

I'm using Hive version 3.1.3 on Hadoop 3.3.4 with Tez 0.9.2. I'm trying to run a SELECT statement on table that Hive created and manages. The query never finishes and fails. The full error message is below, but this appears to be the relevant…
Patrick Tucci
  • 1,824
  • 1
  • 16
  • 22
1
vote
2 answers

Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster

I have installed tez and want to run the example like this hadoop jar tez-examples-0.10.1.jar orderedwordcount /input /output but it's not work and the log is Log Type: stderr Log Upload Time: Thu May 12 13:19:25 +0800 2022 Log Length: 77 Error:…
lvkai
  • 21
  • 3
1
vote
1 answer

hive tez mapr distribution query have issues with tez engine

Query fails on tez but fine on MR. We have tried many different permutations and combinations of set ... paarmeters but unable to run this query successfully in tez. Query in MR runs in about 20 mins but tez engine, we saw that query was consuming…
Sanjay
  • 63
  • 7
1
vote
0 answers

Are the bucket hash algorithms of tez and MR different?

I'm using Hive 3.1.2 and tried to create a bucket with bucket version=2. When I created a bucket and checked the bucket file using hdfs dfs -cat, I could see that the hashing result was different. Are the hash algorithms of Tez and MR different?…
alwaysnoob
  • 33
  • 4
1 2
3
…
12 13