3

Exception in thread "main"

java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support.
at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65)
at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:193)
at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:178)
at org.apache.hadoop.io.compress.CompressionCodec$Util.createInputStreamWithCodecPool(CompressionCodec.java:157)
at org.apache.hadoop.io.compress.SnappyCodec.createInputStream(SnappyCodec.java:164)

I could see snappy library is installed properly. I have mentioned lib path using -Djava.library.path, still resulting in same error.

Any suggestions please?

hadoop checknative -a

Native library checking:

hadoop:  true /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
snappy:  true /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/lib/native/libsnappy.so.1
lz4:     true revision:10301
bzip2:   true /lib64/libbz2.so.1
openssl: true /usr/lib64/libcrypto.so
dokgu
  • 4,957
  • 3
  • 39
  • 77
ShriRam
  • 31
  • 1

2 Answers2

0

Looks like you're running into:

Snappy may complain Hadoop is built without snappy if libhadoop is not found.

https://issues.apache.org/jira/browse/HADOOP-13684

You should debug for the libhadoop.so and look at your LD_LIBRARY_PATH for starters.

tk421
  • 5,775
  • 6
  • 23
  • 34
0

Issue resolved.

I was using incorrect version of hadoop-core jar file in my classpath.

After modifying to use the current CDH version of hadoop core jar, was able to perform snappy compression and decompression.

ShriRam
  • 31
  • 1