circe doesn't see field when it contains an array

Question

I've got 2 "tests", of which the one where I'm trying to decode a user works, but the one where I'm trying to decode a list of users doesn't:

import User._
import io.circe._
import io.circe.syntax._
import io.circe.parser.decode

class UserSuite extends munit.FunSuite:
  test("List of users can be decoded") {

    val json = """|{
                  |   "data" : [
                  |       {
                  |         "id" : "someId",
                  |         "name" : "someName",
                  |         "username" : "someusername"
                  |       },
                  |       {
                  |         "id" : "someId",
                  |         "name" : "someName",
                  |         "username" : "someusername"
                  |       }
                  |   ]
                  |}""".stripMargin    
    println(decode[List[User]](json))
  }

  test("user can be decoded") {
    val json = """|{
                  |   "data" : {
                  |         "id" : "someId",
                  |         "name" : "someName",
                  |         "username" : "someusername"
                  |       }
                  |}""".stripMargin
    println(decode[User](json))
  }

The failing one produces

Left(DecodingFailure(List, List(DownField(data))))

despite the fact that both the json's relevant structure and the decoders (below) are the same.

final case class User(
    id: String,
    name: String,
    username: String
)

object User:
    given Decoder[List[User]] = 
        deriveDecoder[List[User]].prepare(_.downField("data"))
    
    given Decoder[User] = 
        deriveDecoder[User].prepare(_.downField("data"))

As far as I understand this should work, even according to one of Travis' older replies but it doesn't.

Is this a bug? Am I doing something wrong?

For reference, This is Scala 3.2.0 and circe 0.14.1

Try with `Decoder[List[User]].prepare(_.downField("data"))` rather than `deriveDecoder` — Luis Miguel Mejía Suárez, Nov 16 '22 at 20:56
@LuisMiguelMejíaSuárez that's giving me `Left(DecodingFailure(Missing required field, List(DownField(data), DownArray, DownField(data))))` which is understandable, my user decoder won't work like this. So basically it works. When I declare the decoder as a value in the test and pass it explicitly. But when I'm trying to use this definition in the User object though, that results in an infinite loop (which the complier warns me for) — TomPoczos, Nov 16 '22 at 21:28
Ah right, that makes sense. TBH, I would rather do the `downField` on the place rather than on the decoders. — Luis Miguel Mejía Suárez, Nov 16 '22 at 22:01

score 1 · Accepted Answer · answered Nov 17 '22 at 07:09

The thing is that that you need two different encoders for User, the one expecting data field to decode the 2nd json and the one not expecting data field while deriving decoder for a list. Otherwise the 1st json should be

"""|{
   |   "data" : [
   |       {
   |          "data" : 
   |              {
   |                 "id" : "someId",
   |                 "name" : "someName",
   |                 "username" : "someusername"
   |              }
   |       },
   |       {
   |          "data" : 
   |              {
   |                 "id" : "someId",
   |                 "name" : "someName",
   |                 "username" : "someusername"
   |              }
   |       }
   |   ]
   |}""

It's better to be explicit now

final case class User(
                       id: String,
                       name: String,
                       username: String
                     )

object User {
  val userDec: Decoder[User] = semiauto.deriveDecoder[User]
  val preparedUserDec: Decoder[User] = userDec.prepare(_.downField("data"))

  val userListDec: Decoder[List[User]] = {
    implicit val dec: Decoder[User] = userDec
    Decoder[List[User]].prepare(_.downField("data"))
  }
}

val json =
  """|{
     |   "data" : [
     |       {
     |         "id" : "someId",
     |         "name" : "someName",
     |         "username" : "someusername"
     |       },
     |       {
     |         "id" : "someId",
     |         "name" : "someName",
     |         "username" : "someusername"
     |       }
     |   ]
     |}""".stripMargin

decode[List[User]](json)(User.userListDec)
// Right(List(User(someId,someName,someusername), User(someId,someName,someusername)))


val json1 =
    """|{
       |   "data" : {
       |         "id" : "someId",
       |         "name" : "someName",
       |         "username" : "someusername"
       |       }
       |}""".stripMargin

decode[User](json1)(User.preparedUserDec)
// Right(User(someId,someName,someusername))

thanks for your effort answering this. Unfortunately, using Scala and circe versions that I'm using, this doesn't seem to be the issue it gets stuck on. I've tried your version and it gets stuck. But it gets stuck on the outermost data field, same as before. The problem that you're pointing out is present in my code, but the original version doesn't get far enough to hit hat issue, and neither does yours (on my laptop). What I suspect is that it works for you because you use Scala 2. From that I'm deducing that this is a circe bug. If you agree with this conclusion I would open a bug report... — TomPoczos, Nov 17 '22 at 10:05
nah, I mixed up the snippets I pasted from you reply. was still just drinking my morning coffee, sorry for the confusion. Let me take another look — TomPoczos, Nov 17 '22 at 10:23
@TomPoczos *"What I suspect is that it works for you because you use Scala 2"* No, it works in Scala 3 as well https://scastie.scala-lang.org/DmytroMitin/fDXGB598ShqqlusZ67pOIA — Dmytro Mitin, Nov 17 '22 at 13:52

circe doesn't see field when it contains an array

1 Answers1