Decode a list of IO[Int] in http4s when returning value in response

Question

I am currently trying to write extension for external API using http4s Client and Server tools. My idea was: when endpoint, which I created using Server is triggered, make multiple requests (to same endpoint on external API, but with different parameters) using Client, do some operations on the data received in every of this request (for example, sum all Int fields from data), and then unite result of operations in single list and return this list. Since all request results is wrapped in IO, return value for method I created is List[IO[Int]], but I cant find how to decode it to use in response when endpoint is triggered.

I can sucsessfully return in responce single IO[Int] using the following encoder:

implicit val intEntityEncoder: EntityEncoder[IO, Int] = jsonEncoderOf[IO, Int]

I thought that since IO[Int] encoder is declared, I can declare simply:

implicit val testEncoder: EntityEncoder[IO, List[IO[Int]]] = jsonEncoderOf[IO, List[IO[Int]]]

But it gives me error instead:

No given instance of type io.circe.Encoder[List[cats.effect.IO[Int]]] was found for an implicit parameter of method jsonEncoderOf in trait CirceInstances.
I found:

    io.circe.Encoder.encodeIterable[cats.effect.IO[Int], List](
      io.circe.Encoder.encodeIterable[Int, cats.effect.IO](
        io.circe.Encoder.encodeInt
      , /* missing */summon[cats.effect.IO[Int] => Iterable[Int]])
    , ???)

But no implicit values were found that match type cats.effect.IO[Int] => Iterable[Int].
  implicit val testEncoder: EntityEncoder[IO, List[IO[Int]]] = jsonEncoderOf[IO, List[IO[Int]]]

Jasper-M · Accepted Answer · 2023-05-01T12:52:12.030

3

You can't encode a List[IO[Int]] into a single response. An IO[Int] is essentially a program that can somehow result in an Int. So you have a list of multiple programs that each result in Int. What you probably want is to end up with one IO[List[Int]]. Then you have a program that results in a list, and that list can be encoded in a single response.

The most straightforward solution is to sequence your list of IO.

import cats.syntax.all._

val list: List[IO[Int]] = ???
val result: IO[List[Int]] = list.sequence

Cleaner is to not end up with a list of IO in the first place.

import cats.syntax.all._

def performRequest(params: Params): IO[Int] = ???
val paramss: List[Params] = ???

val result: IO[List[Int]] = paramss.traverse(performRequest)

Note that if you would like to perform all these requests in parallel, then that is as easy as simply using parSequence or parTraverse instead.

edited May 01 '23 at 12:52

answered May 01 '23 at 12:40

Jasper-M

14,966
2
26
37

Thanks, that is exactly what I needed, refactored my code using `traverse` and it works like a charm! If I may ask one more question, is there any way to execute this requests in batches? It seems like external api cant handle more than 3 consecutive requests, so if `params` is bigger than 3 entities, my request fails. – Alexey Gavrilik May 01 '23 at 13:33
You can use `list.parTraverseN(3)(foo)` if you import `cats.effect.syntax.all._`, or `IO.parTraverseN(3)(list)(foo)`. That will guarantee that at most 3 requests will run concurrently. – Jasper-M May 01 '23 at 13:41

Decode a list of IO[Int] in http4s when returning value in response

1 Answers1