0

Trying to run this code

Main.java:

public static void main(String[] args) {
...
..
Properties properties = new Properties();
AppProps.setApplicationJarClass(properties, Main.class);
HadoopFlowConnector flowConnector = new HadoopFlowConnector(properties);

Scheme sourceScheme = new queries.ParquetTupleScheme(new Fields("a", "b", "c"));
Tap inTap = new Hfs(sourceScheme, inPath);
...
...
...
}

And i'm getting this error:

java.lang.NoClassDefFoundError: cascading/scheme/Scheme

Here is what i tried so far

  1. When I replace this:

    Scheme sourceScheme = new ParquetTupleScheme(new Fields("a", "b", "c"));

    with this:

    Scheme sourceScheme = null;

    the error goes away

  2. When I'm creating class that extends Scheme<JobConf, RecordReader, OutputCollector, Object[], Object[]> like ParquetTupleScheme
    the error goes away

  3. When I'm trying to check if this is a specific parquet-cascading error

    Object a = new PigCombiner()

    the error goes away

I'm using :

  • cascading 2.5.1
  • parquet-cascading 1.3.0
  • hadoop-core 1.2.1

What i'm doing wrong?

matteodv
  • 3,992
  • 5
  • 39
  • 73
hdmi3killer
  • 89
  • 1
  • 1
  • 12

1 Answers1

0

I have encountered the same problem before. As far as I know, this problem stems from missing cascading Scheme.class in your classpath, thereby throwing this exception during the runtime.

In order to resolve this, you should include the cascading-core-2.5.1.jar into your hadoop classpath by using 'export HADOOP_CLASSPATH=/jar_directory/cascading-core-2.5.1.jar'. You could use 'hadoop classpath' to check whether the jar is properly included.

This is my debut in stackoverflow. :)

Jason Chou
  • 13
  • 3