0

In the Spark GraphX javadoc the fromEdges() method has undocumented arguments like evidence$17. Presumably these are artifacts of the Scala implementation, but what should I do with them in Java?

public static <VD,ED> Graph<VD,ED> fromEdges(RDD<Edge<ED>> edges,
     VD defaultValue,
     StorageLevel edgeStorageLevel,
     StorageLevel vertexStorageLevel,
     scala.reflect.ClassTag<VD> evidence$16,
     scala.reflect.ClassTag<ED> evidence$17)

UPDATE: found some other Scala/Java examples, and it seems like the right thing is ClassTag$.MODULE$.apply(myClass) where myClass is the class of the VD or ED type, respectively. Not that this actually works, as it leads to later, mysterious exceptions, like java.lang.ArrayStoreException: java.lang.Integer deep in org.apache.spark.graphx.impl.EdgePartitionBuilder.toEdgePartition

UPDATE: Indeed, the ClassTag$.MODULE$.apply(Long.class) works for me. The error I was getting was due to passing in 1 for the default value instead of 1L.

Wheezil
  • 3,157
  • 1
  • 23
  • 36
  • Could you please provide a minimal reproducible example? I would expect the scala.reflect.ClassTag.apply(YourVertex.class) and scala.reflect.ClassTag.apply(YourEdge.class) expressions to work as the evidence arguments. The [Calling a Scala from Java Play Framework which takes a ClassTag](https://stackoverflow.com/questions/21582045/calling-a-scala-from-java-play-framework-which-takes-a-classtag) thread seems relevant. – Leonid Vasilev Jun 19 '23 at 21:55
  • Thanks Leonid. Sorry I forgot to follow up, see my latest update. This is solved, except that I'm getting a StackOverflow error from GraphX, but I'll start a new post for that. – Wheezil Jun 20 '23 at 13:52

1 Answers1

0

The evidence arguments are filled in using ClassTag$.MODULE$.apply(myClass) where myClass is the class of the VD or ED type, respectively.

If you use Long.class for the ED and VD types, make sure to use a matching value for the defaults when you call Graph.fromEdges(). In particular for Long.class use e.g. 1L for the default value instead of 1.

Wheezil
  • 3,157
  • 1
  • 23
  • 36