scala type mismatch error in graphX code

Question

I'm new to scala, learning it for apache-spark. I wrote a simple function in scala for graphX

def foo(edge: EdgeTriplet[Map[Long, Double], Double]): Iterator[(VertexId, Map[Long, Double])] = {

    val m = edge.srcAttr

    for((k, v) <- m){
        if (v + edge.attr < edge.dstAttr.getOrElse(k, 10.0))
            Iterator(edge.dstId, Map(k -> v + edge.attr))

        else

            Iterator.empty

    }

}

Errors

Name: Compile Error
Message: <console>:37: error: type mismatch;
 found   : Double
 required: String
                   Iterator(edge.dstId, Map(k -> v + edge.attr))
                                                          ^
<console>:35: error: type mismatch;
 found   : Unit
 required: Iterator[(org.apache.spark.graphx.VertexId, Map[Long,Double])]
    (which expands to)  Iterator[(Long, Map[Long,Double])]
           for((k, v) <- m){
                      ^
StackTrace:

Why scala is treating v as string? and what is the cause of second error?

After editing the code as suggested by @Alexey, I'm getting error

Name: Compile Error
Message: <console>:30: error: type mismatch;
 found   : scala.collection.immutable.Map[org.apache.spark.graphx.VertexId,scala.collection.immutable.Map[Long,Double]]
    (which expands to)  scala.collection.immutable.Map[Long,scala.collection.immutable.Map[Long,Double]]
 required: Iterator[(org.apache.spark.graphx.VertexId, Map[Long,Double])]
    (which expands to)  Iterator[(Long, Map[Long,Double])]
                 (k, v) <- edge.srcAttr
                        ^
StackTrace:

If it helps, I'm implementing a different version of the sendMessage function from this code

Alexey Romanov · Answer 1 · 2017-09-12T13:26:57.733

1

The first is a common problem with + in Scala, unfortunately. In this case you have not k -> (v + edge.attr), as you probably expected, but (k -> v) + edge.attr. And the only + method on a Tuple2 accepts String. To fix it, just add the correct parentheses.

The second error is because for(...) { ... } returns Unit (it's translated to a foreach call). You are missing yield. In fact if you want to use for, it should be something like

for {
  (k, v) <- m
  x <- if (v + edge.attr < edge.dstAttr.getOrElse(k, 10.0))
      Iterator((edge.dstId, Map(k -> (v + edge.attr))))
    else
      Iterator.empty[(Long, Map[Long,Double])]
} yield x

I'd prefer to write it as

m.flatMap { case (k, v) => 
  if (v + edge.attr < edge.dstAttr.getOrElse(k, 10.0))
    Iterator((edge.dstId, Map(k -> (v + edge.attr))))
  else
    Iterator.empty[(Long, Map[Long,Double])]
}

edited Sep 12 '17 at 13:26

answered Sep 12 '17 at 07:59

Alexey Romanov

167,066
35
309
487

First error has resolved , thanks to your solution. Still searching solution for second – pauli Sep 12 '17 at 08:18
please can you write the full code of the solutions to the second problem because I'm getting error mismatch found: Iterable[Any], required Iterator[(Long, Map[Long,Double])] – pauli Sep 12 '17 at 12:55
Edited again (hopefully this should be enough...) – Alexey Romanov Sep 12 '17 at 13:28
I'm still getting an error but it's different. So we have made progress. I have edited the question to add error. – pauli Sep 12 '17 at 23:54
Look at `Map`'s scaladoc. – Alexey Romanov Sep 13 '17 at 07:41

scala type mismatch error in graphX code

1 Answers1