Stack Overflow
Stack Overflow

Questions tagged [hadoop3]

Use for questions specific to Apache Hadoop 3.0 features (i.e. Erasure Coding, YARN Timeline Service v2, Opportunistic Containers, 3+ NameNode fault-tolerance). For general questions related to Apache Hadoop use the tag [hadoop].

112 questions
3
votes
1 answer

Hadoop/HDFS 3.1.1 (on Java 11) Web UI crash when loading the file explorer?

After start-dfs.sh, I can navigate to http://localhost:9870. The NameNode seems to be running just fine. Then I click on "Utilities -> Browse the file system" and I get this prompted in the web browser: Failed to retrieve data from…
Martin Andersson
  • 18,072
  • 9
  • 87
  • 115
2
votes
0 answers

Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/default: java.net.ConnectException: Connection refused (state=08S01,code=0

hadoop 3.3.1 Hive 2.3.9 I use embedded database. hive-site.xml: javax.jdo.option.ConnectionURL jdbc:derby:;databaseName=metastore_db;create=true JDBC connect string for a JDBC…
Venus
  • 1,184
  • 2
  • 13
  • 32
2
votes
1 answer

Is it possible to use Hadoop 3.x and Hive 3.x using spark 2.4?

We use spark 2.4.0 to connect to Hadoop 2.7 cluster and query from Hive Metastore version 2.3. But the Cluster managing team has decided to upgrade to Hadoop 3.x and Hive 3.x. We could not migrate to spark 3 yet, which is compatible with Hadoop 3…
Saiteja Parsi
  • 412
  • 1
  • 5
  • 15
2
votes
2 answers

Hadoop : Permission denied (publickey,password, keyboard-interactive)

While installing Hadoop I got many errors but this one just doesn't go. No matter what I do, it keeps popping again and again. As soon as I am starting Hadoop by the command ./start-all.sh, I get the error: localhost: rajneeshsahai@localhost:…
Rajneesh Sahay
  • 21
  • 1
  • 3
2
votes
1 answer

Could not run jar file in hadoop3.1.3

I tried this command in command prompt (run as administrator): hadoop jar C:\Users\tejashri\Desktop\Hadoopproject\WordCount.jar WordcountDemo.WordCount /work /out but i got this error message: my application got stopped. 2020-04-04 23:53:27,918…
Teja_naik
  • 35
  • 1
  • 6
2
votes
0 answers

Hadoop 3.2.1 and HBase 2.2.3 incompatability: ERROR in hadoop-functions.sh

I am running Hadoop 3.2.1 and Hbase 2.2.3 on Centos8. I downloaded Hadoop from an Apache mirror following the Apache documentation precisely. Hadoop works just fine. I then downloaded HBase from an Apache mirror and followed the Apache documentation…
SGTAMS
  • 21
  • 2
2
votes
2 answers

Hadoop namenode formatting windows - java.lang.UnsupportedOperationException

I am in a databasing class at school and my professor is having us work with hadoop v3.2.1. In following a youtube tutorial to install on windows, I am stuck on the formatting namenode part. this is what comes up in cmd: 2020-03-15 15:38:05,819 INFO…
Pairnatus
  • 35
  • 5
2
votes
4 answers

How to write a table to hive from spark without using the warehouse connector in HDP 3.1

when trying to use spark 2.3 on HDP 3.1 to write to a Hive table without the warehouse connector directly into hives schema using: spark-shell --driver-memory 16g --master local[3] --conf spark.hadoop.metastore.catalog.default=hive val df =…
Georg Heiler
  • 16,916
  • 36
  • 162
  • 292
2
votes
1 answer

Spark not able to read Erasure coded parquet files in Hadoop 3

I have built Hadoop 3.2.0 on a RHEL 6.7 linux box using Intel ISA-L library. Also, enabled the native library support in the Hadoop installation. I have copied some parquet format files on this test cluster using "haoop fs - copyFromLocal" with…
jcools
  • 71
  • 4
2
votes
2 answers

Spark does not use correct configuration from core-site.xml

When I try to read a parquet file from a specified location like /test with spark.read.parquet() i get an error saying file://test does not exist. When I add the core-site.xml as a resource in code with sc.hadoopConfiguration.addResource(new…
Hypnotise
  • 99
  • 1
  • 5
2
votes
1 answer

Is it possible to run Spark (2.3) jobs on hadoop3 clusters specifically HDP 3.1 and CDH6 (beta)

Also, CDH 6 is in beta stage and do they support spark 2.3 without any bells and whistles? is it possible to run the same old spark 2.x versions (2.3 specifically) on hadoop 3 enabled CDH or Hadoop clusters? I'm interested in knowing the backwards…
Suraj N
  • 78
  • 10
2
votes
1 answer

hadoop3 can not create timeline server hbase table

I read the Hadoop 3 about the timeline server v.2 document, it says Finally, run the schema creator tool to create the necessary tables: bin/hadoop org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -create The…
user1208081
  • 1,057
  • 4
  • 15
  • 29
2
votes
0 answers

Hadoop exception while starting

Exception while running sudo sbin/start-dfs.sh /usr/local/Cellar/hadoop/3.1.0/libexec/bin/../libexec/hadoop-functions.sh: line 398: syntax error near unexpected token <' …
Mohsin
  • 469
  • 5
  • 13
2
votes
1 answer

How to upgrade Hadoop Cluster from 2.6.0 to 3.0.0?

The current version of out hadoop cluster is 2.6.0-CDH5.10.0, and We want to upgrade to 3.0.0. There are significant differences between the two versions, such as node ports being inconsistent, and so on. How to upgrade the Hadoop Cluster from 2.6.0…
xuzq_zander
  • 21
  • 2
2
votes
0 answers

Error while running hadoop commands "Unrecognized option -". Could not create Java Virtual Machine

I am trying to create hadoop 3.1 cluster on Ubuntu 18 machine. I am following the steps given here Hadoop cluster steps and currently I setup only one machine but when I tried to run commands like "hadoop version" or "hdfs namenode -format" I am…
hemant A
  • 185
  • 14
1
2
3 4 5 6 7 8