Illegal Access error in GraphX

Question

I'm using Spark and Graphx on IntelliJ IDEA for the first time. I'm trying to create a graph and running queries on it but I'm getting the following error:

java.lang.IllegalAccessError: tried to access class org.apache.spark.util.collection.Sorter from class org.apache.spark.graphx.impl.EdgePartitionBuilder

Here's my code:

package org.apache.spark.examples
// import required spark classes


import org.apache.spark._
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import org.apache.spark.graphx._
import org.apache.spark.rdd.RDD
// define main method (scala entry point)
object HelloWorld {
  def main(args: Array[String]) {

    val conf = new SparkConf().setAppName("HelloWorld")
    val sc = new SparkContext(conf)
    val vertexArray = Array(
      (1L, ("Alice", 28)),
      (2L, ("Bob", 27)),
      (3L, ("Charlie", 65)),
      (4L, ("David", 42)),
      (5L, ("Ed", 55)),
      (6L, ("Fran", 50))
    )
    val edgeArray = Array(
      Edge(2L, 1L, 7),
      Edge(2L, 4L, 2),
      Edge(3L, 2L, 4),
      Edge(3L, 6L, 3),
      Edge(4L, 1L, 1),
      Edge(5L, 2L, 2),
      Edge(5L, 3L, 8),
      Edge(5L, 6L, 3)
    )
    val vertexRDD: RDD[(Long, (String, Int))] = sc.parallelize(vertexArray)
    val edgeRDD: RDD[Edge[Int]] = sc.parallelize(edgeArray)
    val graph: Graph[(String, Int), Int] = Graph(vertexRDD, edgeRDD)

    // Solution 1
    graph.vertices.filter { case (id, (name, age)) => age > 30 }.collect.foreach {
      case (id, (name, age)) => println(s"$name is $age")
    }

    // Solution 2
    graph.vertices.filter(v => v._2._2 > 30).collect.foreach(v => println(s"${v._2._1} is ${v._2._2}"))

    // Solution 3
    for ((id,(name,age)) <- graph.vertices.filter { case (id,(name,age)) => age > 30 }.collect) {
      println(s"$name is $age")
    }

    sc.stop()

  }
}

What versions are you compiling with and running on? I suspect that this is probably a classpath issue. — Rohan Aletty, Oct 11 '15 at 12:35
scala version: 2.10.6 Spark version: 1.5.1 on hadoop 2.6 Maven version: 3.3.0 — CMWasiq, Oct 11 '15 at 16:13
I don't know. I didn't do anything Maven specific accept setting MAVEN_HOME because for some reason, I wasn't able to create Scala packages without it — CMWasiq, Oct 12 '15 at 08:15
So are you building using SBT? If you are using Maven, take a look at `mvn dependency:tree -Dverbose` to see where the conflicting packages are and perhaps add an exclusion on the artifact whose dependency you do not want. For SBT, you can exclude transitive dependences using either `exclude()` or an exclusion rule. — Rohan Aletty, Oct 12 '15 at 17:33
The issue has been resolved. The problem was with the scala version. Thanks for your comments! — CMWasiq, Oct 13 '15 at 08:36

score 1 · Answer 1 · answered Oct 13 '15 at 08:38

1

The problem was with the scala version I was using, 2.10.6, because I had read from an installation guide that spark worked well only with 2.10.x versions. But after installing the latest version, 2.11.x, everything worked fine

answered Oct 13 '15 at 08:38

CMWasiq

79
10

Illegal Access error in GraphX

1 Answers1