Update edge weight in graphx

Question

I'm playing around with graphx. I've built a graph I'm trying to update the weight of a relation,

import org.apache.spark.rdd.RDD
import org.apache.spark.graphx._
def pageHash(title:String )  = title.toLowerCase.replace(" ","").hashCode.toLong


val vertexArray = Array(
  (pageHash("Alice"), ("Alice")),
(pageHash("Bob"), ("Bob")),
(pageHash("Charlie"), ("Charlie")),
(pageHash("David"), ("David")),
(pageHash("Ed"), ("Ed")),
(pageHash("Fran"), ("Fran"))
)     
val edgeArray = Array(
 Edge(pageHash("Bob"), pageHash("Alice"), 7),
 Edge(pageHash("Bob"), pageHash("David"), 2),
Edge(pageHash("Charlie"), pageHash("Bob"), 4),
Edge(pageHash("Charlie"), pageHash("Fran"), 3),
Edge(pageHash("David"), pageHash("Alice"), 1),
Edge(pageHash("Ed"), pageHash("Bob"), 2),
Edge(pageHash("Ed"), pageHash("Charlie"), 8),
Edge(pageHash("Ed"), pageHash("Fran"), 3)
)    


val vertexRDD: RDD[(Long, (String))] = sc.parallelize(vertexArray)
val edgeRDD: RDD[Edge[Int]] = sc.parallelize(edgeArray)
val graph: Graph[(String), Int] = Graph(vertexRDD, edgeRDD)

graph.triplets.filter(triplet => triplet.srcAttr.equals("Bob")&&triplet.dstAttr.equals("Alice")).collect()

graph.triplets.filter(triplet => triplet.srcAttr.equals("Bob")&&triplet.dstAttr.equals("Alice")).
    map(triplet=> triplet.attr.toString.toInt+1).collect()

I'm not able to increase the weight of the node , is there any way to do this?

What do you expect to happen here? Or in other words what you mean by _I'm not able to do this_? Best thing is always [https://stackoverflow.com/help/mcve](https://stackoverflow.com/help/mcve) — zero323, Feb 25 '16 at 23:36
it's not about the imports. Where do you define your graph value ? — eliasah, Mar 01 '16 at 09:31
I am still not sure what is the issue here. Can you provide expect and actual output? — zero323, Mar 01 '16 at 11:10
@zero323 I want to update the weight of the edge for instance in this case for source node "Bob" and destination node"Alice" the weight should increase to 8 from 7 within same RDD PS:I know RDDs are immutable, — tourist, Mar 01 '16 at 11:36

score 5 · Accepted Answer · answered Apr 02 '16 at 10:20

You can't directly update an Edge, but you can functionally do the same thing by adding a new Edge with the same src and dst to the edges RDD in your Graph and the delta of the weight, and then calling groupEdges on your. In other words, if you have the following graph:

val edges = sc.parallelize(Array(Edge(1L, 2L, 1.0), Edge(2L, 3L, 2.0)))
val vertices = sc.parallelize(Array((1L, "Bob"), (2L, "Tom"), (3L, "Jerry")))

val graph = Graph(vertices, edges)

You can add 1.0 to the weight of one of the edges like this:

val newGraph = Graph(graph.vertices, graph.edges.union(
  sc.parallelize(Array(Edge(2L, 3L, 1.0)))
).groupEdges((a, b) => a + b)

How efficient is the Union operation as in if I'm trying to add an edge iteratively 1billion times? My current understanding is it will take new rdd with an edge throwing away the previous graph generated in each iteration. Is it an efficient operation in this scenario? — tourist, Apr 02 '16 at 12:17

Update edge weight in graphx

1 Answers1