How to cast Any Scala to Double

Question

I am iterating a dataframe and need to apply some custom code to each row so I'm doing this

convertedPaths.foreach { row : Row => 
  cFMap = row.getValuesMap(channelSet.toSeq)

Assume that channelSet is a set of all column names. Now I've declared cFMap is of type [String, Any] as getValuesMap (as i understood would return the data type of the column)

Also, all the columns are of Long type so I was trying to do this :

channelSet.foreach { key : String => 
  var frequency = cFMap.get(key).get.asInstanceOf[Double]

  var value = c * frequency

given c is a variable of type Double and value needs to be a product of c and frequency but that gives me the following error :

overloaded method value * with alternatives: (x: Double)Double <and> (x: Float)Double <and> (x: Long)Double <and> (x: Int)Double <and> (x: Char)Double <and> (x: Short)Double <and> (x: Byte)Double cannot be applied to (Any)

Why is asInstanceOf[Double] not a correct solution and what could be the solution for this?

Where does `c` come from in the statement `var value = c * frequency` and what is the type of `c`? — Jesper, May 28 '16 at 14:29
See: http://stackoverflow.com/q/33007840/1560062, but your code cannot work correctly anyway, if convertedPaths is a `DataFrame`. — zero323, May 28 '16 at 14:42
@zero323 convertedPaths is indeed a DataFrame and that's what I'm more keen to know why is tah twe can't use it this way cause not doing so would leave me with a messy workaround to undertake — hbabbar, May 28 '16 at 14:49
https://spark.apache.org/docs/latest/programming-guide.html#understanding-closures-a-nameclosureslinka — zero323, May 28 '16 at 15:08

score 0 · Answer 1 · answered May 31 '16 at 04:37

Although this doesnt completely answer my problem statement and I'm trying to read up on Accumulators on how to collect my result but the major portion of the logic can be run inside the convertedPaths.forEach loop.

I was able to resolve my problem by assigning

var cFMap = Map[String, Long]()

instead of

var cFMap = Map[String, Any]()

as i know all columns in convertedPaths are Long type

How to cast Any Scala to Double

1 Answers1