JSON processing in scala

Question

I am in the process of switching from json to avro format and all my unit tests are still expecting json. I can convert the newly avro files to json but the schema is embedded. For example:

val j = """{"field1": "someString", "field2": 1607359485}"""
// now becomes:
val jAvro = """{"field1": {"string":"someString"}, "field2": {"long":1607359485}}"""

I want, momentarily, to remove the schema from jAvro. Parsing it is not a problem:

import play.api.libs.json.{ JsObject, Json }
Json.parse(jAvro).as[JsObject]

But how can I programmatically remove all the schema information, so that I get the same outpout as Json.parse(j).as[JsObject]?

EDIT

Thanks for your answers. I forgot to mention that the level of intricacy might be superior to my example.

I could have:

val jAvro = """{"field1": {"string":"someString"}, "field2": {"long":1607359485}, "field3":{"SomeObject":{"subfield1":{"int":11},"subfield2":{"string":"someString2"}}}"""

and I would like to extract:

val j = """{"field1":"someString", "field2":1607359485, "field3":{"subfield1":11,"subfield2":"someString2"}}"""

Please read [What should I do when someone answers my question?](https://stackoverflow.com/help/someone-answers) — Tomer Shetah, Jan 19 '21 at 21:43

Tomer Shetah · Answer 1 · 2020-12-08T11:12:30.233

1

You can try something like:

def updateJson(json: JsObject): Map[String, Any] = json.value.view.mapValues(_.validate[JsObject] match {
  case JsSuccess(value, _) =>
    value.fields.headOption.map(_._2).map {
      case obj: JsObject =>
        updateJson(obj)
      case value: JsValue =>
        value
      case other =>
        ??? // Error handling
    }.getOrElse(???) // Error handling
  case JsError(errors) =>
    println("Do here error handling", errors)
    ???
}).toMap

Code run at Scastie.

edited Dec 08 '20 at 11:12

answered Dec 07 '20 at 17:40

Tomer Shetah

8,413
7
27
35

Thanks, it works with the question I asked, but please see my EDIT – fricadelle Dec 08 '20 at 10:27
@fricadelle, do you have more options/requirements? Or is that it? – Tomer Shetah Dec 08 '20 at 10:46
It's pretty much it. I can have also null fields that we can discard. The "arity" (depth of json fields) is never more than two. jAvro = """{"field1": {"string":"someString"}, "field2": {"long":1607359485}, "field3":{"SomeObject":{"subfield1":{"int":11},"subfield2":{"string":"someString2"}}, "field4": null}""" – fricadelle Dec 08 '20 at 10:50
@fricadelle, I've update my answer. Next time try to include all the requirements in your post. Please read the [tour](https://stackoverflow.com/tour), and [How do I ask a good question?](https://stackoverflow.com/help/how-to-ask). – Tomer Shetah Dec 08 '20 at 11:13
In my version of scala (2.10) there is no mapValues method for json.value.view – fricadelle Dec 08 '20 at 13:06
@fricadelle, 2.10 is really old. Please see here a run in 2.10: https://scastie.scala-lang.org/toshetah/lFgCS54RRI6Bn2st4JVRFw/9 – Tomer Shetah Dec 08 '20 at 13:09
Hello, thanks for your work, I'm almost there, it still breaks on null and when I have List type subfields. Example : https://scastie.scala-lang.org/93Pa0BYrSjKKDTAW8xXQCA – fricadelle Dec 08 '20 at 13:53

score 0 · Answer 2 · answered Dec 07 '20 at 18:30

You can use Reads.mapReads:

import play.api.libs.json._

implicit val valueAsObj: Reads[JsValue] = Reads[JsValue] {
  case JsObject(fields) => fields.headOption match {
    case Some((_/*type*/, value)) =>
      JsSuccess(value)

    case _ =>
      JsError("error.singlefield.expected")
  }

  case _ =>
    JsError("error.jsobject.expected")
}

val mapReads: Reads[Map[String, JsValue]] = Reads.mapReads(valueAsObj)
val objReads: Reads[JsObject] = mapReads.map(JsObject(_))

Then:

val jAvro = """{"field1": {"string":"someString"}, "field2": {"long":1607359485}}"""

Json.parse(jAvro).validate(objReads)
// JsSuccess({"field1":"someString","field2":1607359485},)

JSON processing in scala

2 Answers2