Transforming JSON with state in circe

Question

Note: I'm copying this question over from the circe Gitter channel for the sake of posterity.

Suppose we want to translate this JSON document:

{
  "places": [{
    "id": "dadcc0d9-0615-4e46-9df4-2619f49930a0"
  }, {
    "id": "21d02f4b-7e88-47d7-bf2b-48e50761b6c3"
  }],
  "transitions": [{
    "id": "10a3aee5-541b-4d04-bb45-cb1dbbfe2128",
    "startPlaceId": "dadcc0d9-0615-4e46-9df4-2619f49930a0",
    "endPlaceId": "21d02f4b-7e88-47d7-bf2b-48e50761b6c3"
  }],
  "routes": [{
    "id": "6ded1763-86c0-44ce-b94b-f05934976a3b",
    "transitionId": "10a3aee5-541b-4d04-bb45-cb1dbbfe2128"
  }]
}

Into this:

{
  "places": [{
    "id": "1"
  }, {
    "id": "2"
  }],
  "transitions": [{
    "id": "3",
    "startPlaceId": "ref:1",
    "endPlaceId": "ref:2"
  }],
  "routes": [{
    "id": "4",
    "transitionId": "ref:3"
  }]
}

I.e., we want to replace the UUID in every id with a simple incremented numeric identifier, and to replace all references to each UUID with references to the new identifiers.

How can we do this with circe?

score 11 · Accepted Answer · answered Apr 13 '16 at 17:41

It's possible to accomplish this relatively straightforwardly with the state monad transformer from Cats (a library that circe depends on):

import cats.data.StateT
import cats.std.option._
import cats.std.list._
import cats.syntax.traverse._
import io.circe.{ Json, JsonObject }
import java.util.UUID

def update(j: Json): StateT[Option, Map[UUID, Long], Json] = j.arrayOrObject(
  StateT.pure[Option, Map[UUID, Long], Json](j),
  _.traverseU(update).map(Json.fromValues),
  _.toList.traverseU {
    case ("id", value) => StateT { (ids: Map[UUID, Long]) =>
      value.as[UUID].toOption.map { uuid =>
        val next = if (ids.isEmpty) 1L else ids.values.max + 1L
        (ids.updated(uuid, next), "id" -> Json.fromString(s"$next"))
      }
    }
    case (other, value) => value.as[UUID].toOption match {
      case Some(uuid) => StateT { (ids: Map[UUID, Long]) =>
        ids.get(uuid).map(id => (ids, other -> Json.fromString(s"ref:$id")))
      }
      case None => update(value).map(other -> _)
    }
  }.map(Json.fromFields)
)

And then:

import io.circe.literal._

val doc: Json = json"""
{
  "places": [{
    "id": "dadcc0d9-0615-4e46-9df4-2619f49930a0"
  }, {
    "id": "21d02f4b-7e88-47d7-bf2b-48e50761b6c3"
  }],
  "transitions": [{
    "id": "10a3aee5-541b-4d04-bb45-cb1dbbfe2128",
    "startPlaceId": "dadcc0d9-0615-4e46-9df4-2619f49930a0",
    "endPlaceId": "21d02f4b-7e88-47d7-bf2b-48e50761b6c3"
  }],
  "routes": [{
    "id": "6ded1763-86c0-44ce-b94b-f05934976a3b",
    "transitionId": "10a3aee5-541b-4d04-bb45-cb1dbbfe2128"
  }]
}
"""

And finally:

scala> import cats.std.long._
import cats.std.long._

scala> import cats.std.map._
import cats.std.map._

scala> update(doc).runEmptyA
res0: Option[io.circe.Json] = 
Some({
  "places" : [
    {
      "id" : "1"
    },
    {
      "id" : "2"
    }
  ],
  "transitions" : [
    {
      "id" : "3",
      "startPlaceId" : "ref:1",
      "endPlaceId" : "ref:2"
    }
  ],
  "routes" : [
    {
      "id" : "4",
      "transitionId" : "ref:3"
    }
  ]
})

If any id field isn't a legitimate UUID, or if any other field contains a reference to an unknown UUID, the computation will fail with None. This behavior could be refined a bit as needed, and if you wanted more specific information about where the error occurred, you could adapt the implementation to work with cursors instead of JSON values (but this would get a little more complex).

Can `ids.updated(uuid, next)` be replaced with `ids + (uuid -> next)`? — Łukasz, Apr 13 '16 at 19:56
@Łukasz Yes, but I find `updated` a little clearer, since it doesn't require the explicit creation of a tuple, and since there will always only be one key-value pair we're adding there. — Travis Brown, Apr 13 '16 at 20:49
All right, thanks for explanation. I find it a little confusing as this would also allow to replace existing value but that will never happen and your intention is to insert a new element, but now I can also see advantages of your approach. — Łukasz, Apr 13 '16 at 23:28
But `+` also replaces existing values. :) Agreed, though—this is a matter of fine differences in taste and I don't think either `+` or `updated` are clearly better. — Travis Brown, Apr 14 '16 at 12:02
Damn, another thing I learned, I wouldn't think that + would do an update, but if I think about it, it makes perfect sense :) — Łukasz, Apr 14 '16 at 12:06

Transforming JSON with state in circe

1 Answers1