4

From the search, I found this question and I had the same problem following the instructions.

CDH4 Hbase using Pig ERROR 2998 java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/filter/Filter

But didn't worked for me. Besides, I'm using Ubuntu and Hadoop-2.2.0, Hbase-0.96.1-hadoop2 and Pig-0.12.0.

Bashrc

export JAVA_HOME=/usr
export HADOOP_INSTALL=/home/ubuntu/hadoop-2.2.0
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL

export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_COMMON_HOME}/lib/native
export HADOOP_OPTS="-Djava.library.path=${HADOOP_COMMON_HOME}/lib"
export PIG_HOME=/home/ubuntu/pig-0.12.0
export HBASE_HOME=/home/ubuntu/hbase-0.96.1-hadoop2
export PIG_CLASSPATH="'${HADOOP_INSTALL}/bin/hbase classpath':$PIG_HOME/bin"

Log Pig

Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error.     
org/apache/hadoop/hbase/filter/WritableByteArrayComparable

java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/filter/WritableByteArrayComparable

Pig.properties

fs.defaultFS=hdfs://hadoop-master/
mapred.job.tracker=hadoop-master:8021
Community
  • 1
  • 1
fsi
  • 1,319
  • 1
  • 23
  • 51

1 Answers1

5

I got it right

export PIG_CLASSPATH=$PIG_HOME/pig-0.12.0-withouthadoop.jar:$HBASE_HOME/hbase-0.94.1.jar:$HBASE_HOME/lib/*:$HADOOP_COMMON_LIB_NATIVE_DIR/lib/*:$PIG_CLASSPATH

fsi
  • 1,319
  • 1
  • 23
  • 51
  • You said you are using Hbase-0.96.1-hadoop2. Then why (and from where) are you adding $HBASE_HOME/hbase-0.94.1.jar to export? I am also stuck at same problem. And I dont have hbase-0.94.1.jar. Should I download it? – Mahesh M Sep 02 '14 at 14:12
  • 1
    You can get it on `http://ftp.unicamp.br/pub/apache/hbase/stable/`. The name you need to look for is `hbase-x-hadoop2`. – fsi Sep 02 '14 at 14:21
  • 1
    Thank you! Any idea how to get the same with pig-0.12 and hbase-0.98.3? The other one worked anyway. Just wanted o use the latest available. – Mahesh M Sep 02 '14 at 14:52
  • You must check the 3 version of the pig vs hadoop vs hbase if they are compatible between. – fsi Sep 02 '14 at 15:00
  • Alright. Guess that will be there in the release notes, right? – Mahesh M Sep 02 '14 at 15:02
  • Yes. But careful, not always the version up to date are the best to use(version problem). – fsi Sep 02 '14 at 22:43
  • i am using hbase-0.98.6.1 version. Now i want to integrate with pig but i can't find the $HBASE_HOME/hbase-0.98.6.1.jar file. Where can i find it or what is the other relevant jar file for that? – Kumar Oct 24 '14 at 06:18
  • It should be on hbase folder. – fsi Jan 05 '15 at 16:20