Scala Doobie not inserting values into database

Question

My code to insert values is:

def insert(link: entity.Link) : IO[Int] = {
    logger.info("Inserting link: " + link.toString())
    sql"insert into links (title,url,publication_date,feed_id) values (${link.getTitle},${link.getUrl},${link.getPublicationDate},${link.getFeed.getId})"
        .update
        .run
        .transact(transactor)
}

The method calling it is:

    val links = for{
        feeds <- getFeeds()
        ls: List[Link] = feeds.flatMap(rssFeedReader.readFeedItems(_).asScala)
        
        ios = ls.map(
            link => 
                linkPersistence
                    .countByUrl(link.getUrl())
                    .map(x => if(x==0) linkPersistence.insert(link) else identity(x))
            
        )

        s <- ios.sequence
    }yield(s)
    val result = links.unsafeRunSync()
    logger.info("Result: " + result.toString())

Unfortunately, I cannot use For Expressions, because IO doesn't have withFilter and I am inside a Maven project (I am migrating a Spring Boot application to Typelevel stack), so I cannot use the improved For Expressions sbt plugin.

My problem is that rows are not inserted into the database. On the log there is:

reddit_bot_1  | 2021-04-23 13:06:57.805  INFO 1 --- [text-global-167] r.i.repository.LinkPersistence           : Inserting link: Link{id=0, feed=Feed{id=79, url='https://ytali.com/feed/'}, title='Se il grillo non fosse stato parlante', url='https://ytali.com/2021/04/21/se-il-grillo-non-fosse-stato-parlante/', publicationDate=Wed Apr 21 11:12:59 GMT 2021}
reddit_bot_1  | 2021-04-23 13:06:57.838  INFO 1 --- [text-global-167] r.i.repository.LinkPersistence           : Inserting link: Link{id=0, feed=Feed{id=82, url='https://cronacheletterarie.com/feed/'}, title='Scrivere come in una serie tv', url='https://www.cronacheletterarie.com/2021/04/22/scrivere-come-in-una-serie-tv-due-nuovi-thriller-italiani/', publicationDate=Thu Apr 22 06:50:30 GMT 2021}
reddit_bot_1  | 2021-04-23 13:06:57.993  INFO 1 --- [nio-8080-exec-1] reddit_bot.service.LinkUpdater           : Result: List(1, 1, 1, 1, 1, 1, 1, IO$970148716, IO$398844220, IO$2051359159, IO$1102460161, IO$1646104176, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, IO$1525243662, 1, 1, 1, 1, 1, 1, 1, 1, 1, IO$183629085, IO$2057291647, IO$1962915922, IO$1686018284, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, IO$1629671523, IO$1703036679, IO$1823891149, IO$2090799695, IO$189359223, IO$2089070694, IO$572184845, IO$638827115, IO$1723402242, IO$1063752499, IO$1262614371, IO$1882901634, IO$1592599455, IO$1561386362, IO$422495060, IO$1071010351, IO$135298961, IO$1395689700, IO$1022084533, IO$824291117, IO$977707631, IO$1835432555, IO$264428575, IO$858145293, IO$1014815708, IO$1654086426, IO$2074886515, IO$2048606753, IO$551801989, IO$2029261832, IO$1640654268, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, IO$1096825326, IO$2132380347, IO$664110340, IO$364966863, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, IO$1429149648, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, IO$1724908444, IO$1835403239, IO$2133511951, IO$1385617069, IO$876405632, IO$1773704159, IO$1078164304, IO$604307019, IO$2004298152, 1, IO$1526495986, 1, 1, 1, 1, 1, 1, 1, 1, 1)

The transactor is created like this:

def transactor() = Transactor.fromDriverManager[IO](
  "com.mysql.cj.jdbc.Driver",
  "jdbc:mysql://database:3306/redditbot",
  "redditbot",
  "redditbot"
)

UPDATE

As suggested by @luis-miguel-mejía-suárez I modified my pom.xml file like this:

         <plugin>
            <groupId>net.alchim31.maven</groupId>
            <artifactId>scala-maven-plugin</artifactId>
            <version>4.4.1</version>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <scalaVersion>${scala.version}</scalaVersion>
                <sourceDir>src/main/scala</sourceDir>
                <compilerPlugins>
                    <compilerPlugin>
                        <groupId>com.olegpy</groupId>
                        <artifactId>better-monadic-for_2.13</artifactId>
                        <version>0.3.1</version>
                    </compilerPlugin>
                </compilerPlugins>
            </configuration>
        </plugin>

But the problem remains.

[**bm4**](https://github.com/oleg-py/better-monadic-for) is a compiler plugin, not a **sbt** plugin. So you can also add it to a **maven** build. Follow the instructions of [**kind-projector**](https://github.com/typelevel/kind-projector) and adapt it to **bm4** _(once you get it running it may be good to open a PR to the **bm4** project to add the **maven** instructions to their `README`)_ - But I am not sure if that would indeed or not fix your problem. - BTW, `map` + `sequence` = `traverse` — Luis Miguel Mejía Suárez, Apr 23 '21 at 13:36
Are you sure statements in `insert` are not another way around? — Krzysztof Atłasik, Apr 23 '21 at 14:13
@KrzysztofAtłasik sorry, I can't understand what you mean. The problem is with the `insert` statements? — Vitaly Olegovitch, Apr 23 '21 at 14:45
@LuisMiguelMejíaSuárez unfortunately, that doesn't work. See this issueI opened: https://github.com/oleg-py/better-monadic-for/issues/52 — Vitaly Olegovitch, Apr 23 '21 at 17:06
@VitalyOlegovitch `if foo` will not work with `IO` not even using **bm4**, **bm4** helps removing unnecessary `withFilter` calls to things like `(a, b) <- IO.pure(3 -> 5)` - BTW, you should not need to also add the plugin as a normal dependency. — Luis Miguel Mejía Suárez, Apr 23 '21 at 17:10
@LuisMiguelMejíaSuárez You are right, you already explained that here: https://stackoverflow.com/questions/55715182/value-withfilter-is-not-a-member-of-cats-io-in-for-comprehension/55716191#comment98111109_55715182 — Vitaly Olegovitch, Apr 23 '21 at 17:28

Vitaly Olegovitch · Accepted Answer · 2021-04-24T18:08:23.280

As written by @LuisMiguelMejíaSuárez

As the error clearly says, IO does not have a withFilter method. (You can check the scaladoc here). When you put the type explicitly, you are basically filtering all elements that match such type. And, since the method does not exists, it won't compile. - And no, I do not know any workaround.

But, I can think on at least ne reason of why it should not have it. IO is not exactly a "container" of elements, like List, since it only is a description of a computation, and if you ant to see it like a container it will only had one element, like Option. But, unlike the former, there is not concept of empty IO. Thus, filtering an IO would not make sense.

The workaround that I have found is moving the filter inside another function :

def insertIfNotPresent(link: Link, foundLinksCount: Int): IO[Option[Int]] = 
    foundLinksCount match{
        case 0 => linkPersistence.insert(link).map(Option(_))
        case _ => IO.pure(None)
    }

The calling code becomes:

for{
  foundLinksCount: Int <- linkPersistence.countByUrl(link.getUrl())
  n: Option[Int] <- insertIfNotPresent(link, foundLinksCount)
}yield(n)

Scala Doobie not inserting values into database

1 Answers1