4

I tried to set up sqoop (sqoop-1.4.3.bin__hadoop-1.0.0) on Ubuntu. I can run the basic sqoop help etc without problems. When I run the following I get an error:

sqoop import --connect jdbc:mysql://localhost/test --table sales -m 1

13/04/19 10:35:24 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/hadoop/hadoop-1.0.4
13/04/19 10:35:24 ERROR orm.CompilationManager: It seems as though you are running sqoop with a JRE.
13/04/19 10:35:24 ERROR orm.CompilationManager: Sqoop requires a JDK that can compile Java code.
13/04/19 10:35:24 ERROR orm.CompilationManager: Please install a JDK and set $JAVA_HOME to use it.
13/04/19 10:35:24 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Could not start Java compiler.
    at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:176)

I googled to find an answer to this error and most answer point to setting JAVA_HOME to your JDK, which I have (in ~/.bashrc [on Ubuntu]).

$ echo $JAVA_HOME
/usr/lib/jvm/jdk1.7.0_15

These are the other variables I set:

export HADOOP_COMMON_HOME=/opt/hadoop/hadoop-1.0.4
export HADOOP_MAPRED_HOME=/opt/hadoop/hadoop-1.0.4
export SQOOP_HOME=/opt/hadoop/sqoop-1.4.3.bin__hadoop-1.0.0
export PATH=$PATH:$SQOOP_HOME/bin

I also added the mysql jdbc driver to the lib folder.

Any ideas why I get this error? Thanks in advance, Diddy

Diddy
  • 193
  • 1
  • 3
  • 11

3 Answers3

7

Scoop creates java classes that represent the schema of the table you are importing in. So to create the classes you need the jdk instead of the jre.

Set the JAVA_HOME in the ./bashrc and the $HADOOP_HOME/conf/hadoop-env.sh as well.

That should work.

Mark Vickery
  • 1,927
  • 3
  • 22
  • 34
  • 1
    Thanks a lot! I had JAVA_HOME set in ./bashrc to JDK, but $HADOOP_HOME/conf/hadoop-env.sh was still referencing JRE, so I changed this. Now everything is working smoothly. – Diddy Apr 21 '13 at 15:31
  • Hi, I am trying to connect sqoop with hadoop. Everytime I try to start sqoop it gives me this error "It seems as though you are running sqoop with a JRE - But JAVA_HOME set to JDK" Hadoop version 2.7.2 Hive version 1.2.1 – KJ_kaka Mar 09 '16 at 14:04
  • @KJ_kaka Have you set it in both places? (Although this might differ from Hadoop 1 in Hadoop 2.) – Mark Vickery Mar 09 '16 at 21:28
  • Thanks alot for the reply @MarkVickery.... HBase, Hive and other components working fine except sqoop. I checked .bashrc-file, hadoop-env.sh file, I even checked in sqoop files. Everything seems to be fine. I even grep Hadoop and Sqoop folders for "JRE" but found nothing.... This is kinda weird error which I am facing... – KJ_kaka Mar 10 '16 at 03:01
1

Sqoop will eventually execute Hadoop shell script, so I would make sure that your Hadoop is configured properly to use the JDK instead of JRE.

Jarek Jarcec Cecho
  • 1,736
  • 1
  • 10
  • 13
0

I had the same issue on Ubuntu.

My $JAVA_HOME was correctly set to /usr/lib/jvm/java-7-openjdk-amd64, but looking into /usr/lib/jvm/java-7-openjdk-amd64/bin I noticed that "javac" was not there. That's what Sqoop needs but could not find.

So the problem was simply that my installation of Java came without Java compiler. The solution was then very simple:

sudo apt-get install openjdk-7-jdk
claudod
  • 815
  • 8
  • 8