How to visualize graph with network in zeppelin?

Question

I want to visualize my graph with %network in zeppelin. I've defined nodes and edges by reading from JSON file.

val nodes = spark.read.option("multiline","true").json("/opt/nodes.json")
val edges = spark.read.option("multiLine","true").json("/opt/edges.json")
val graph = GraphFrame(nods,edges)

I've read the documentation but it isn't good for file. I don't know how can I visualize it.

score 0 · Accepted Answer · answered May 03 '20 at 14:22

Following code, although not very generic theoretically visualize the data from GraphFrame object (don't use it on giant graph). But although it generates correct data, that are visible in UI, there is a JavaScript error during visualization (I've tried 0.9.0-preview1) - I've created ZEPPELIN-4793 to handle it.

import org.graphframes._
import org.apache.spark.sql.functions._

  def graphframeToNetworkText(g: GraphFrame, vertexLabel: String, edgeLabel: String): String = {
    val v = g.vertices
    val vDataFields = v.schema.fields.map{_.name}.filterNot(_ == "id").map(col(_))
    val vJson = v.select(col("id"), lit(vertexLabel).as("label"),
      struct(vDataFields: _*).as("data"))
      .toJSON.collect.mkString("[",", ","]")
    val e = g.edges
    val eDataFields = e.schema.fields.map{_.name}
      .filterNot(x => x == "src" || x == "dst").map(col(_))
    val timeUUID = udf(() => java.util.UUID.randomUUID().toString)
    val eJson = e.select(col("src").as("source"),
      col("dst").as("target"),
      lit(edgeLabel).as("label"),
      struct(eDataFields: _*).as("data"))
      .withColumn("id", timeUUID())
      .toJSON.collect.mkString("[",", ","]")

    val sb = new StringBuilder
    sb.append("%network {")
      .append("\n\"nodes\": ")
      .append(vJson)
      .append(",\n\"edges\": ")
      .append(eJson)
      .append(",\n\"directed\": false,\n\"types\":[\"")
      .append(edgeLabel)
      .append("\"],\n\"labels\":{\"")
      .append(vertexLabel)
      .append("\": \"#3071A9\"}\n}")

    sb.toString()
  }

that could be called as following:

val g: GraphFrame = examples.Graphs.friends
print(graphframeToNetworkText(g, "person", "rel"))

How to visualize graph with network in zeppelin?

1 Answers1