0

I am loading data from phoenix through this:

val tableDF = sqlContext.phoenixTableAsDataFrame("Hbtable", Array("ID", "distance"), conf = configuration)

and want to carry out the following computation on the column values distance: 

val list=Array(10,20,30,40,10,20,0,10,20,30,40,50,60)//list of values from the column distance
val first=list(0)
val last=list(list.length-1)
var m = 0; 
for (a <- 0 to list.length-2) {
  if (list(a + 1) < list(a) && list(a+1)>=0)
  {
     m = m + list(a)
  } 
}
val totalDist=(m+last-first)
Alicia Garcia-Raboso
  • 13,193
  • 1
  • 43
  • 48
vivek
  • 71
  • 8
  • Do you want to do the computations on the `DataFrame` or do you want to collect the column values into an actual `Array` type? – Glennie Helles Sindholt Nov 01 '16 at 10:24
  • Any thing is fine, at the end i want to find the total distance as given in the code. i prefer to carry out the computations on an array as it is easy, you can suggest me even. – vivek Nov 01 '16 at 10:30

1 Answers1

0

You can do something like this. It returns Array[Any]

`val array = df.select("distance").rdd.map(r => r(0)).collect()

If you want to get the data type properly, then you can use. It returns the Array[Int]

val array = df.select("distance").rdd.map(r => r(0).asInstanceOf[Int]).collect()
Shankar
  • 8,529
  • 26
  • 90
  • 159