I have a function like:
def createDataset[T](seq:Seq[T]): Dataset[T] = {
import spark.implicits._
seq.toDS()
}
And this is not compiling, it doesn't find toDS function.
It also doesn't work in this way
def createDataset[T](t:T): Dataset[T] = {
import spark.implicits._
Seq(t).toDS()
}
The case classes that I'm using are
case class Person(id: Long, name: String, age: Int) {}
case class Address(a_id:Long, street:String, number: Int) {}
What I can do to have a generic function that creates a Dataset given a T generic class which is always a case class?
edit:
Solution provided by Terry Dactyl is not working for me and show this error when f function is called
import org.apache.spark.sql.{Dataset, Encoder, SparkSession}
def f[T <: Product : Encoder](s: Seq[T]): Dataset[T] = {
val spark = SparkSession.builder.getOrCreate()
import spark.implicits._
s.toDF.as[T]
}
f(Seq(
Person(1, "John", 25)
Person(2, "Paul", 22)
))
No implicits found for parameter ev$1: Encoder[Person]