Stack Overflow
Stack Overflow

Questions tagged [apache-tez]

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data.

The Apache Tez project is aimed at building an application framework which allows for a complex directed-acyclic-graph of tasks for processing data. It is currently built atop Apache Hadoop YARN

See Hive-on-Tez configuration properties.

192 questions
0
votes
2 answers

org.apache.hadoop.hive.ql.io.orc.OrcStruct cannot be cast to org.apache.hadoop.io.BinaryComparable

at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:563) at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83) ... 17 more Caused by: java.lang.ClassCastException:…
ravi R
  • 11
  • 1
  • 2
0
votes
1 answer

Unable to start hive using tez execution engine

Im using Hadoop 2.7.3 version and hive 1.2.1 version. I face problem with hive using tez engine. Is there any setup error or other kind of error ?? Logging initialized using configuration in…
Rohan
  • 66
  • 1
  • 1
  • 8
0
votes
0 answers

Hive applications are lazy to start

Hive(TEZ) queries are starting in lazy fashion on YARN. We are running Hive queries on Tez engine. As soon as we submit the queries, we are able to see the status as RUNNING, but the actual job is not starting until 10 to 15 mins. I am not very sure…
Bukkasamudram
  • 351
  • 1
  • 2
  • 10
0
votes
1 answer

Are Big Data Jobs Idempotent?

I have seen it stated that jobs are idempotent when we write a big data job using MR, Spark, or Tez as the execution engine. The job first writes data to a temporary directory i.e. ".hivestaging..." or "_temporary" Then data is merged to its final…
dinesh028
  • 2,137
  • 5
  • 30
  • 47
0
votes
1 answer

Suggestions required in increasing utilization of yarn containers on our discovery cluster

Current Setup we have our 10 node discovery cluster. Each node of this cluster has 24 cores and 264 GB ram Keeping some memory and CPU aside for background processes, we are planning to use 240 GB memory. now, when it comes to container set up, as…
akash sharma
  • 411
  • 2
  • 24
0
votes
1 answer

need help in re-writing this query, which uses same data set multiple times, as per explain plan

We have a query run by our development team that's heavy in resources and looking at the explain plan, it looks like its uses the same data set multiple times. Is there anyway we can re-write this query. Now, i tried to replace the co-related query…
Kumar
  • 119
  • 10
0
votes
2 answers

How do I fix this Kryo exception when using a UDF on hive?

I have a hive query that worked in hortonworks 2.6 sandbox, but it doesn't work on sandbox ver. 3.0 because of this exception: Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 95 …
pavel_orekhov
  • 1,657
  • 2
  • 15
  • 37
0
votes
0 answers

Why does hive on tez run twice as fast sometimes?

I've run some experiments with hive on tez. I have a query that 80% of the time runs for about 45 seconds and the rest of the time runs for about 24 seconds. What could that rare speedup be attributed to?
pavel_orekhov
  • 1,657
  • 2
  • 15
  • 37
0
votes
1 answer

tez session not getting started in hive

I am new to hadoop and yarn and tried running a hive insert query on hive 2 beeline. The Info I am getting is: INFO : Tez session hasn't been created yet. Opening session and my system seemed to be hanged. I tried checking for the logs on YARN UI…
coder_enthusiast
  • 33
  • 7
0
votes
1 answer

How to run a query on the view in Hive?

We are running a simple select on the view (which contains lot of data) and we are getting "GC overhead limit exceeded, out of memory error. We want to run this query so that report that runs on top of this view can work. It runs on Tez. The query…
akash sharma
  • 411
  • 2
  • 24
0
votes
2 answers

How can I fix java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 3.1.0?

I get a java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 3.1.0 exception in my query. Here's the query: WITH t1 as (select * from browserdata join citydata on cityid=id), t2 as (select uap.device as device, uap.os as…
pavel_orekhov
  • 1,657
  • 2
  • 15
  • 37
0
votes
0 answers

Hive query vertex failure in tez mode of execution

I'm trying to execute a Hive query -- Select a,b,c,d,e,f,cast(g as timestamp) - cast(f as timestamp) as runtime from table ORDER BY d,e desc limit 100 It is falling with below error TaskAttempt 1 failed, info=[Error: Failure while running…
user3686069
  • 91
  • 6
  • 15
0
votes
1 answer

Hive configuration is not reflected

I am using hive-site.xml to change some of the configuration parameters. for ex. hive.execution.engine tez But when I connect to the hive server using beeline with the following command -…
Vijay Muvva
  • 1,063
  • 1
  • 17
  • 31
0
votes
1 answer

What does VIRTUAL_MEMORY_BYTES task counter mean in Hadoop?

The following excerpt from The Definitive Guide provides high level details as shown below but what exactly is virtual memory is referring to in this task counter? How to interpret it? How is it related to PHYSICAL_MEMORY_BYTES? Following is an…
Aravind Yarram
  • 78,777
  • 46
  • 231
  • 327
0
votes
1 answer

Hive insert query failing with error return code -101

I am trying to run a simple insert statement as below: insert into table `bwc_test` partition(call_date) select * from `bwc_master`; Then it fails with the below error: INFO : Loading data to table dtc.bwc_test partition (call_date=null) from…
Andy Reddy
  • 93
  • 2
  • 9
1 2 3
…
12 13