Play-JSON cannot find implicit Writes[T] for type parameter T

Question

I'm new to Scala and still learning about most of the features. I'm trying to convert a Dataset[T] to Json. I'm using json play to create implicit Writes. Type parameters work fine when I load my Dataset:

def processEvent[T](spark: SparkSession, inputPath: String)(implicit encoder: Encoder[T]): Unit = {
  val ds = spark.read.parquet(inputPath).as[T]
  ds.collect().foreach { event =>
    val serializedEvent = eventToJson[T](spark, event)
    postEvent(serializedEvent)
  }
}

But in the call to EventToJson, I get the error: No Json serializer found for type T. Try to implement an implicit Writes or Format for this type.

def eventToJson[T](spark: SparkSession, event: T): String = {
  Json.toJson(event).toString()
}

When I replace the parameterized type with one of my case classes, the code works fine:

def eventToJson(spark: SparkSession, event: MyCaseClass): String = {
  Json.toJson(event).toString()
}

Why doesn't the parameterized type find the corresponding case class and implicit Writes?

I would recommend to read some tutorial about [Scala implicit](https://docs.scala-lang.org/tour/implicit-parameters.html) [Play JSON](https://www.playframework.com/documentation/2.8.x/ScalaJsonCombinators) — cchantep, Sep 23 '20 at 18:38

Dmytro Mitin · Answer 1 · 2020-09-23T17:39:13.623

Try to constrain T with a context bound:

def eventToJson[T: Writes](spark: SparkSession, event: T): String = {
  Json.toJson(event).toString()
}

or

def eventToJson[T: Format](spark: SparkSession, event: T): String = {
  Json.toJson(event).toString()
}

Then probably you'll have to add the same context bound to processEvent

import org.apache.spark.sql.{Encoder, SparkSession}
import play.api.libs.json.{Json, Writes}

def processEvent[T: Encoder: Writes](spark: SparkSession, inputPath: String): Unit = {
  val ds = spark.read.parquet(inputPath).as[T]
  ds.collect().foreach { event =>
    val serializedEvent = eventToJson[T](spark, event)
    postEvent(serializedEvent)
  }
}

def eventToJson[T: Writes](spark: SparkSession, event: T): String = {
  Json.toJson(event).toString()
}

Why doesn't the parameterized type find the corresponding case class and implicit Writes?

Because it is defined for the case class, not for T (even though T can become the type of case class when you apply eventToJson).

In Scala if a definition of method foo produces compile error that an implicit of type Bar is not found then you can try to add that implicit to the signature of foo