Spark Java Error :lambda expressions are not supported in -source 1.5

Question

Got below code from http://spark.apache.org. I am getting below error....

Code:

JavaRDD<String> lines = sc.textFile(logFile);
JavaPairRDD<String, Integer> pairs = lines.mapToPair(s -> new Tuple2(s, 1));

Error :

lambda expressions are not supported in -source 1.5 (use -source 8 or higher to enable lambda expressions)

but below code works perfectly fine. Can you please help me to find the reason?

New Code:

JavaRDD<String> lines = sc.textFile(logFile);
JavaPairRDD<String, String> prodPairs = lines.mapToPair(new PairFunction<String, String, String>() {
            public Tuple2<String, String> call(String s) {
                String[] prodSplit = s.split(",");
                return new Tuple2<String, String>(prodSplit[2], prodSplit[0]+","+prodSplit[1]+","+prodSplit[2]); 
            }
        });

@Vijay K: What version of Java you are using? Lambdas are introduced in Java 8,if you already have Java 8, you can change the Java version and compiler details in Eclipse IDE by going to project build path. — Shankar, Nov 16 '16 at 16:43

Rodrigo Menezes · Answer 1 · 2021-01-07T18:38:47.927

Are you using maven?

By default maven use source and target compilation to java 1.5:

Also note that at present the default source setting is 1.5 and the default target setting is 1.5, independently of the JDK you run Maven with. If you want to change these defaults, you should set source and target as described in Setting the -source and -target of the Java Compiler. Source

You need to configure in pom build compiler plugin:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.2</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
    </plugins>
</build>

correct ending build tag. From to – parneeti sood Jan 07 '21 at 08:18 — parneeti sood, Jan 07 '21 at 08:18

Shankar · Answer 2 · 2016-11-17T12:05:51.857

0

What version of Java you are using? Lambda expressions(->) are introduced in Java 8,if you use Java less than 8, then its not going to work.

If you already have Java 8, you can change the Java version and compiler details in Eclipse IDE by going to project build path and select the correct compiler version and jdk version.

If you are running maven from command line, you can set Java home first, then run the build.

For Example:

set JAVA_HOME=<JDK Location>

then

mvn clean install

edited Nov 17 '16 at 12:05

answered Nov 17 '16 at 02:17

Shankar

8,529
26
90
159

Thanks Shankar. New to java. java --version is showing : java version "1.8.0_111". not sure why maven is taking old version. is there any setting which i need to change? – Vijay K Nov 17 '16 at 08:53
@VijayK: Updated my answer. – Shankar Nov 17 '16 at 12:08

score 0 · Answer 3 · answered Nov 17 '16 at 02:56

You might want to read Lambda Expression docs. This was added in JDK 1.8. The arrow operator usage in the first piece of your code is a Lambda Expression. It seems that you are compiling your code with Java 1.5. Therefore you are rightly getting the error. In the second piece of code, you have replaced the Lambda expression with the traditional code. Therefore the compiler stopped complaining. Unless you have a good reason, you might want to upgrade your compiler to 1.8. You are missing out on a lot of good features that were added in subsequent versions of the JDK.

Spark Java Error :lambda expressions are not supported in -source 1.5

Code:

Error :

New Code:

3 Answers3