1

I'm trying to use fs2 streams 0.10.0-M9 and doobie version 0.5.0-M9 for getting a sequence of objects from an http call which I want to then insert into a postgres database but I'm having issues structuring this code, getting the following error:

Error:(49, 12) Cannot prove that Seq[fs2.Stream[cats.effect.IO,Int]] <:< fs2.Stream[cats.effect.IO,O2]. .join(100)

What i want to do is run the insert statements concurrently once the call to the web service returns. Here's the code:

fetchProducts(date).map{items  =>
        items.map( i =>
          Stream.eval(upsertProductIO(i).transact(xa))
        )
      }
      .join(100)
      .run
      .unsafeRunSync()

//rest call
def fetchProducts(changeDate: String): Stream[IO, Seq[Product]] = {
//rest call here
}

//DAO code
def upsertProductIO(p: Product): ConnectionIO[Int] = {
  upsertProduct(p).run
}
Jose H. Martinez
  • 161
  • 2
  • 8

1 Answers1

2

The problem is that you have a Seq[Stream[IO, Seq[Product]], What you want, rather, is a Stream[IO, Seq[Product]], which you can do with a Foldable type class instance:

import cats.implicits._
import scala.concurrent.ExecutionContext.Implicits.global

fetchProducts(date).map { items =>
   items.map(i => Stream.eval(upsertProductIO(i).transact(xa)))
}.map(seqOfStream.toList.sequence)
 .join(100)
 .run
 .unsafeRunSync()
Yuval Itzchakov
  • 146,575
  • 32
  • 257
  • 321
  • Thanks. I modified the code as you suggested but I'm getting a similar error: `fetchProducts("2017-12-10T06:17:37:557-0400").map { items => items.map(i => Stream.eval(upsertProductIO(i).transact(ProductDao.xa))) }.map(is => is.toList.sequence) .join(100) .run .unsafeRunSync()` Error:(38, 27) Cannot prove that fs2.Stream[cats.effect.IO,Int] <:< G[A]. }.map(is => is.toList.sequence) – Jose H. Martinez Jan 04 '18 at 17:33
  • @JoseH.Martinez, what version of the Scala compiler do you use? Yuval's code compiles for me with `scalaVersion := "2.11.11"` and `scalacOptions += "-Ypartial-unification"` and fails without `-Ypartial-unification` with an error quite similar to yours. – SergGr Jan 04 '18 at 18:12
  • I'm on scalaVersion := "2.12.4" – Jose H. Martinez Jan 05 '18 at 14:25
  • Did you try compiling with sbt? – Yuval Itzchakov Jan 05 '18 at 14:30
  • @JoseH.Martinez ? – Yuval Itzchakov Jan 06 '18 at 19:12
  • Yes, I'm using sbt. I noticed the problem exists when I change def upsertProductIO(p: Seq[Product]) to def upsertProductIO(p: Product) – Jose H. Martinez Jan 09 '18 at 16:50
  • what's the proper way to accomplish this using upsertProductIO(p: Product) @YuvalItzchakov – Jose H. Martinez Jan 10 '18 at 14:51
  • @JoseH.Martinez Do you still return a `Stream[IO, Seq[Product]]` from `fetchProducts`? – Yuval Itzchakov Jan 10 '18 at 15:00
  • Yes. I wonder if I can take that sequence and then run multiple inserts for each product. Or, make fetchProducts return the sequence of products as a stream rather than a sequence. – Jose H. Martinez Jan 10 '18 at 18:36