JSON decode nested field as Map[String, String] in Scala using circe

Question

A circe noob here. I am trying to decode a JSON string to case class in Scala using circe. I want one of the nested fields in the input JSON to be decoded as a Map[String, String] instead of creating a separate case class for it.

Sample code:

import io.circe.parser
import io.circe.generic.semiauto.deriveDecoder

case class Event(
  action: String,
  key: String,
  attributes: Map[String, String],
  session: String,
  ts: Long
)
case class Parsed(
  events: Seq[Event]
)

Decoder[Map[String, String]]

val jsonStr = """{
  "events": [{
      "ts": 1593474773,
      "key": "abc",
      "action": "hello",
      "session": "def",
      "attributes": {
          "north_lat": -32.34375,
          "south_lat": -33.75,
          "west_long": -73.125,
          "east_long": -70.3125
      }
  }]
}""".stripMargin

implicit val eventDecoder = deriveDecoder[Event]
implicit val payloadDecoder = deriveDecoder[Parsed]
val decodeResult = parser.decode[Parsed](jsonStr)
val res = decodeResult match {
  case Right(staff) => staff
  case Left(error) => error
}

I am ending up with a decoding error on attributes field as follows:

DecodingFailure(String, List(DownField(north_lat), DownField(attributes), DownArray, DownField(events)))

I found an interesting link here on how to decode JSON string to a map here: Convert Json to a Map[String, String]

But I'm having little luck as to how to go about it.

If someone can point me in the right direction or help me out on this that will be awesome.

While you certainly can, what do you prefer to have a **Map[String, String]** instead of a proper case class? Or are the attributes dynamic? — Luis Miguel Mejía Suárez, Jul 07 '20 at 17:10
I am trying to convert this case class into an array of another case class which has key and value fields. I have tried converting the case class of attributes into a map in Scala before but it turns out to be ugly. So I was looking for a way where I can read this object as a Map[String,String] from JSON itself. — Arun Shyam, Jul 07 '20 at 17:13
I do not understand what you meant with: _"I am trying to convert this case class into an array of another case class which has key and value fields"_ - Why do you want to turn a case class into a **Map**? My original question remains, what does a **Map** gives you that a case class doesn't? — Luis Miguel Mejía Suárez, Jul 07 '20 at 17:15
It's something that I need to do Luis. It feels easier for me to convert a Map[String, String] tp a list of case classes of type Attribute(key: String, value: String) rather than converting one case class into a list of some other case class given some of the fields may or may not be present in the attributes that are coming in this JSON. To answer your question, yes, these fields are dynamic and unfortunately I at this point can't ask our partner to change the traffic pattern. — Arun Shyam, Jul 07 '20 at 17:19
Ok so with that context it is clear what you need to do. I would do this, first use **List[Attribute]** instead of **Map[String, String]** on your case class. Define your own explicit decoder for **List[Attribute]** and then use that to automatically derive the decoder of **Event**. I am on mobile right now, so I can't help with the code, but if when I got access to a computer you still haven't receive an answer I will give it a shoot. — Luis Miguel Mejía Suárez, Jul 07 '20 at 17:24
Thank you Luis. Let me give it a shot. While I have an answer for converting it to a map now, but getting each individual field in attributes to a key value pair will be super as I won't need to reconvert again. — Arun Shyam, Jul 07 '20 at 17:30

score 2 · Accepted Answer · answered Jul 07 '20 at 17:13

2

Let's parse the error :

DecodingFailure(String, List(DownField(geotile_north_lat), DownField(attributes), DownArray, DownField(events)))

It means we should look in "events" for an array named "attributes", and in this a field named "geotile_north_lat". This final error is that this field couldn't be read as a String. And indeed, in the payload you provide, this field is not a String, it's a Double.

So your problem has nothing to do with Map decoding. Just use a Map[String, Double] and it should work.

answered Jul 07 '20 at 17:13

C4stor

8,355
6
29
47

Amazing! Thank you for the quick response. This worked for me. Apologies for the stupid mistake. – Arun Shyam Jul 07 '20 at 17:21
2

It's ok :-) Circe errors get some getting used to, but they are in fact quite formidable tools even if they look a bit ugly ! – C4stor Jul 07 '20 at 17:23
@ArunShyam so you said the attributes are dynamic but are you sure they will always be numbers? I thought the idea of a **Map[String, String]** was because the values could also be anything that you would need to parse again latter according to their names. – Luis Miguel Mejía Suárez Jul 07 '20 at 17:33
Yes Luis they are dynamic but currently they are all coming in as double. You are right this may or may not be the case in the future which will definitely cause issues. – Arun Shyam Jul 07 '20 at 17:36

Luis Miguel Mejía Suárez · Answer 2 · 2020-07-08T15:34:25.013

So you can do something like this:

final case class Attribute(
    key: String,
    value: String
)

object Attribute {
  implicit val attributesDecoder: Decoder[List[Attribute]] =
    Decoder.instance { cursor =>
      cursor
        .value
        .asObject
        .toRight(
          left = DecodingFailure(
            message = "The attributes field was not an object",
            ops = cursor.history
          )
        ).map { obj =>
          obj.toList.map {
            case (key, value) =>
              Attribute(key, value.toString)
          }
        }
    }
}

final case class Event(
    action: String,
    key: String,
    attributes: List[Attribute],
    session: String,
    ts: Long
)

object Event {
  implicit val eventDecoder: Decoder[Event] = deriveDecoder
}

Which you can use like this:

val result = for {
  json <- parser.parse(jsonStr).left.map(_.toString)
  obj <- json.asObject.toRight(left = "The input json was not an object")
  eventsRaw <- obj("events").toRight(left = "The input json did not have the events field")
  events <- eventsRaw.as[List[Event]].left.map(_.toString)
} yield events

// result: Either[String, List[Event]] = Right(
//   List(Event("hello", "abc", List(Attribute("north_lat", "-32.34375"), Attribute("south_lat", "-33.75"), Attribute("west_long", "-73.125"), Attribute("east_long", "-70.3125")), "def", 1593474773L))
// )

You can customize the Attribute class and its Decoder, so their values are Doubles or Jsons.

This is super Luis! Certainly generic and robust. I updated my code to work on these lines. — Arun Shyam, Jul 08 '20 at 18:41

JSON decode nested field as Map[String, String] in Scala using circe

2 Answers2