Coerce type "double" to a integer in sparkR

Question

I have a DataFrame in sparkR 'u' which conains ID = 1 1 1 1... and age = 21 23 33 21 ... To take the sum of of 'ages' I do this

sumage<-agg(u, ages="sum")

Now sumage is a DataFrame with type double. I want to have sumage as a integer so I try this

as.numeric(sumage)

but I get this message: "sumu<- agg(u, amount_spent="sum") cannot coerce type 'S4' to vector of type 'double'"

What can be done about this ?

score 1 · Accepted Answer · answered Aug 19 '15 at 13:55

There are two possible solutions. As sumage is actually a DataFrame with one item, you can easily collect it and use as.integer

localSumage <- collect(sumage)[1,1]

which is a numeric type. If needed you can convert it to an integer with

localSumage <- as.integer(localSumage)

Or you can convert the type to an integer in SparkR with

sumage$intergerAge <- cast(sumage[[1]],"integer")

But be aware that it still is a DataFrame with an integer column where your sum is stored.

1 Answers1