10

I am a beginner with hadoop and trying to install and run hadoop in my Ubuntu as a single node cluster. This is my JAVA_HOME in my hadoop_env.sh

# The java implementation to use.
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386/
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}

But when I run it the following errors come-

Starting namenodes on [localhost]
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.
Starting secondary namenodes [0.0.0.0]
0.0.0.0: Error: JAVA_HOME is not set and could not be found.

How do I remove this error?

ayushman999
  • 511
  • 3
  • 6
  • 16

8 Answers8

19

I debugged the code and found out that even though JAVA_HOME is set in the environment, the value is lost as ssh connections to other hosts is made inside the code, and the JAVA_HOME variable that was showing well set in start-dfs.sh became unset in hadoop-env.sh.

The solution to this problem will be to set JAVA_HOME variable in hadoop-env.sh and it should work properly.

Sohil Jain
  • 331
  • 3
  • 6
  • As for @toobee, this worked for me, but why. I'm not a Linux pro, so why is JAVA_HOME not set when SSHing to the local machine? I have mine set in my ~/.profile – Greg B Aug 26 '15 at 07:43
  • That was a bug in the code of Hadoop. The value of JAVA_HOME gets overriden inside the code with something else, even if you have it set in ~/.profile. – Sohil Jain Apr 02 '16 at 18:39
  • Very accurate. Just as an alternative solution you can also set it in /etc/environment and it would work!:) – Prakhar Agrawal Sep 24 '17 at 16:27
15

I had the same error and solved it with Soil Jain's remark, but to make it even a bit more clear: the hadoop-env.sh uses an expression such as

export JAVA_HOME=${JAVA_HOME}

if you hard-code the path to your JVM installation it works

export JAVA_HOME=/usr/lib/jvm/java...

this resolution by environmental variable as is seems to fail. Hard-coding fixed the problem for me.

toobee
  • 2,592
  • 4
  • 26
  • 35
  • Thank you so much! – Halil İbrahim Oymacı Jun 03 '17 at 14:50
  • I used to set a hard-code like `/usr/lib/jdk1.8.0_231` but dataNode restart fail get `resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh su hdfs -l -s /bin/bash -c 'ulimit -c unlimited ; /usr/hdp/3.1.4.0-315/hadoop/bin/hdfs --config /usr/hdp/3.1.4.0-315/hadoop/conf --daemon stop datanode'' returned 1. ERROR: JAVA_HOME /usr/lib/jdk1.8.0_231 does not exist.` – wyx Jan 13 '20 at 03:41
6

Under your HADOOP_HOME/conf directory please update the hadoop-env.sh file. It has entry to export JAVA_HOME.

Setting to appropriate JAVA_HOME in this file should solve your issue.

Ankur Shanbhag
  • 7,746
  • 2
  • 28
  • 38
  • 2
    I have updated it as export JAVA_HOME=/usr/lib/jvm/java-7-sun .. still same error – ayushman999 Dec 17 '13 at 09:56
  • On my case. Started Hadoop successfully after removing java_home variable in user bash profile and adding entry above mentioned in haddop-env.sh file. one info: In latest version of Hadoop, location of hadoop-env.sh file has been changed. i.e. HADOOP_HOME/etc/hadoop/hadoop-env.sh – Ramgau May 20 '18 at 11:43
2

Are you loading hadoop_env.sh? you may be refering to hadoop-env.sh ( dash instead of underscore - that is under conf directory)

BTW, This is a very useful guide for quick installation :

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

griffon vulture
  • 6,594
  • 6
  • 36
  • 57
2

First, you must set JAVA_HOME in your hadoop_env.sh . (your local JAVA_HOME in .bashrc would likely to be ignore somehow)

# The java implementation to use.
export JAVA_HOME=/usr/lib/jvm/default-java

Then, set HADOOP_CONF_DIR point to directory of your hadoop_env.sh . In ~/.bashrc, add the following line:

HADOOP_CONF_DIR="/usr/local/hadoop/etc/hadoop"
export HADOOP_CONF_DIR

Where /usr/local/hadoop/etc/hadoop is the directory contained hadoop_env.sh

Haha TTpro
  • 5,137
  • 6
  • 45
  • 71
1

I'm using hadoop 2.8.0. Even though I exported JAVA_HOME (I put it in .bashrc), I still caught this error while trying to run start-dfs.sh.

user@host:/opt/hadoop-2.8.0 $ echo $JAVA_HOME
<path_to_java>
user@host:/opt/hadoop-2.8.0 $ $JAVA_HOME/bin/java -version
java version "1.8.0_65"
...
user@host:/opt/hadoop-2.8.0 $ sbin/start-dfs.sh
...
Starting namenodes on []
localhost: Error: JAVA_HOME is not set and could not be found.
localhost: Error: JAVA_HOME is not set and could not be found.

The only way I could get it to run was to add JAVA_HOME=path_to_java to etc/hadoop/hadoop-env.sh and then source it:

:/opt/hadoop-2.8.0 $ grep JAVA_HOME etc/hadoop/hadoop-env.sh
#export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=path_to_java
user@host:/opt/hadoop-2.8.0 $ source etc/hadoop/hadoop-env.sh

Maybe that (sourcing hadoop-env.sh) was implied in the posts above. Just thought someone should say it out loud. Now it runs. I've encountered other issues (due, I suspect, to the limited resources on the server I'm using), but at least I got past this one.

mule.ear
  • 66
  • 3
0

Above answers should work as long as you are using default conf directory $HADOOP_HOME/conf or $HADOOP_HOME/etc/hadoop. Here are a few things you should do if you're using a different conf folder.

  1. Copy the hadoop-env.sh file from the default conf directory to your conf folder, say /home/abc/hadoopConf.

  2. Replace the line

    #export JAVA_HOME=${JAVA_HOME}

    with the following:

    export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export HADOOP_CONF_DIR=/home/abc/hadoopConf

Change the values appropriately. If you have any other environment variables related to hadoop configured in your .bashrc or .profile or .bash_profile consider adding them next to the above lines.

Jo Kachikaran
  • 562
  • 1
  • 6
  • 12
-1

It not know space between Program and Files: "Program Files". So, I copy folder of jdk to C: or folder which not contains space in name of folder and assign: export JAVA_HOME=Name_Path_Copied. I see it run ok

hoang
  • 1