Stack Overflow
Stack Overflow

Questions tagged [hadoop3]

Use for questions specific to Apache Hadoop 3.0 features (i.e. Erasure Coding, YARN Timeline Service v2, Opportunistic Containers, 3+ NameNode fault-tolerance). For general questions related to Apache Hadoop use the tag [hadoop].

112 questions
1
vote
0 answers

hadoop 3.3.1 show job history error: Exception in thread "main" java.lang.IllegalArgumentException: JobId string : /output_dir is not properly formed

hadoop 3.3.1 I have successfully run a program hadoop jar units.jar com.clx.bigdata.ProcessUnits /input_dir /output_dir from the printed message,I get the job id:job_1625033931379_0001 I can get job history list from web page of…
Venus
  • 1,184
  • 2
  • 13
  • 32
1
vote
1 answer

hadoop Web UI localhost:50070 can not open

Ubuntu 16.04.1 LTS Hadoop 3.3.1 I try to set up hadoop pseudo distributed mode referring to one network tutorial.and follow below steps. Step 1:setting Hadoop 1.add below code to /etc/profile. export HADOOP_HOME=/home/hadoop/hadoop export…
Venus
  • 1,184
  • 2
  • 13
  • 32
1
vote
1 answer

Does Hadoop 3 support Mahout?

I was wondering if any Mahout version has been confirmed to work properly with any version of Hadoop 3.x. It looks like both Cloudera's and Amazon's Hadoop distribution removed Mahout when they went from Hadoop 2 to Hadoop 3. But I cannot find any…
Jens Roderus
  • 181
  • 9
1
vote
1 answer

Hadoop trying to use JDK install directory as executable command

I am new to Hadoop, and attempting to get the first simple 'Word count' example to run. I have the same problem that was reported here (but the responses there don't resolve the problem): Could not run jar file in hadoop3.1.3 Java is installed to…
Martin Peck
  • 11
  • 1
1
vote
2 answers

Command line too long on maven when building hadoop from source in windows 10

I am trying to build hadoop from source as explained in this article. When building Apache common, everything fails with this error message: command line too long. So, Here's what I have tried(I will update this when I try more): As said in…
user13550497
1
vote
2 answers

Exception in thread "main" org.apache.hadoop.ipc.RemoteException(java.io.IOException) for hadoop 3.1.3

I am trying to run a mapreduce job but I am getting error for Hadoop-3.1.3 hadoop jar WordCount.jar WordcountDemo.WordCount /mapwork/Mapwork /r_out Error 2020-04-04 19:59:11,379 INFO client.RMProxy: Connecting to ResourceManager at…
Teja_naik
  • 35
  • 1
  • 6
1
vote
0 answers

Hadoop 3.2 Oozie -Pig issue

I am using Hadoop 3.2.1 with pig 0.17.0 and oozie 5.1.0. While Ensuring the Pig Workflow sample in Oozie I have encountered an issue in oozie which are mentioned below. I currently using Guava jar 20.0 but I have changed the guava jar…
Yugendhar
  • 11
  • 2
1
vote
0 answers

Where/how are pid files being named in hadoop 3?

During migration from hadoop2 to hadoop3 we noticed a couple of issues with pid files and init scripts. It boils down to the following issue: The upstream init/systemd scripts expect a pid file name like "yarn-yarn-nodemanager.pid", while our Hadoop…
leitmedium
  • 23
  • 5
1
vote
2 answers

HDFS fails to start with Hadoop 3.2 : bash v3.2+ is required

I'm building a small Hadoop cluster composed of 2 nodes : 1 master + 1 worker. I'm using the latest version of Hadoop (3.2) and everything is executed by the root user. In the installation process, I've been able to hdfs namenode -format. Next step…
David Guyon
  • 2,759
  • 1
  • 28
  • 40
1
vote
1 answer

API or Catalog operations for creating Hive External table using HiveWarehouseSession.session(spark).build()

I'm using HDP 3.X cluster and running spark sql using spark_llap, Is there a way to create external hive table using hive.createTable because the example provided in Hortonworks website is to use following code whereas this code will create manged…
Sridhar
  • 27
  • 3
1
vote
0 answers

placement of blocks in hadoop 3

I am currently concerned with block placement where we are trying to implement our own placement algorithm in Hadoop 3. In Hadoop 2, we can find a file called PlacementMonitor.java where we can feed our placement policy but I can't find any such…
dragonfly
  • 11
  • 3
1
vote
0 answers

Yarn of hadoop 3.1.1 can't schedule GPU resource

I have installed Hadoop 3.1.1 which supports GPU resources. And I configured it according to the official document . However, when I run the word count program with a 800M+ file, the GPU finding resource never are used; at the same time the program…
Vic
  • 11
  • 1
1
vote
0 answers

Hortonworks HDP 3 : Error starting ResourceManager

I have installed a new cluster HDP 3 using ambari 2.7. The problem is that resource manager service is not starting. I get the following error: Traceback (most recent call last): File…
Prashant Gupta
  • 21
  • 3
1
vote
1 answer

Hadoop: bad execution compiling WordCount

I have installed Hadoop 3.1.1 and it is working. However, when I try to compile the WordCount example, I am receiving this error: /usr/local/hadoop/libexec/hadoop-functions.sh: line 2358: HADOOP_COM.SUN.TOOLS.JAVAC.MAIN_USER: bad…
CGG
  • 253
  • 1
  • 11
  • 22
1
vote
2 answers

How to enable LLAP in Hive 3?

I installed Hive 3.1.0 on top Hadoop 3.1.1(without using Ambari). How can I enable LLAP feature in Hive.
Nandu
  • 151
  • 1
  • 2
  • 12
1 2
3
4 5 6 7 8