In Scala what is the easiest way to parse json and map to objects?

Question

I'm looking for a super simple way to take a big JSON fragment, that is a long list with a bunch of big objects in it, and parse it, then pick out the same few values from each object and then map into a case class.

I have tried pretty hard to get lift-json (2.5) working for me, but I'm having trouble cleanly dealing with checking if a key is present, and if so, then map the whole object, but if not, then skip it.

I absolutely do not understand this syntax for Lift-JSON one bit:

case class Car(make: String, model: String)

...

val parsed = parse(jsonFragment)
val JArray(cars) = parsed / "cars"

val carList = new MutableList[Car]
for (car <- cars) {
    val JString(model) = car / "model"
    val JString(make) = car / "make"

    // i want to check if they both exist here, and if so 
    // then add to carList
    carList += car
}

What on earth is that construct that makes it look like a case class is being created left of the assignment operator? I'm talking about the "JString" part. Also how is it supposed to cope with the situation where a key is missing?

Can someone please explain to me what the right way to do this is? And if I have nested values I'm looking for, I just want to skip the whole object and go on to try to map the next one.

Is there something more straightforward for this than Lift-JSON?

Would using extractOpt help?

I have looked at this a lot: https://github.com/lift/framework/tree/master/core/json

and it's still not particularly clear to me.

Help is very much appreciated!!!!!

I am a little confused - are you looking to pick out a few values for each item to populate your case classes with that subset, or are you looking to use those few values to perform some tests on and then populate the case object with the entire record? — jcern, Jun 17 '13 at 15:15
@jcern I'm just trying to pick out a few values and populate case classes with subset. — jpswain, Jun 17 '13 at 17:03

score 3 · Accepted Answer · answered Jun 17 '13 at 18:38

Since you are only looking to extract certain fields, you are on the right track. This modified version of your for-comprehension will loop through your car structure, extract the make and model and only yield your case class if both items exist:

for{ 
  car <- cars
  model <- (car \ "model").extractOpt[String]
  make <- (car \ "make").extractOpt[String]
} yield Car(make, model)

You would add additional required fields the same way. If you want to also utilize optional parameters, let's say color - then you can call that in your yield section and the for comprehension won't unbox them:

for{ 
  car <- cars
  model <- (car \ "model").extractOpt[String]
  make <- (car \ "make").extractOpt[String]
} yield Car(make, model, (car \ "color").extractOpt[String])

In both cases you will get back a List of Car case classes.

extractOpt does not work for me... i get "could not find implicit value for parameter formats: net.liftweb.json.Formats" anything I can do to fix this? — jpswain, Jun 18 '13 at 01:39
Ah, I had not added "implicit val formats = net.liftweb.json.DefaultFormats" — jpswain, Jun 18 '13 at 01:46

pagoda_5b · Answer 2 · 2013-06-17T08:34:49.303

The weird looking assignment is pattern-matching used on val declaration.

When you see

val JArray(cars) = parsed / "cars"

it extracts from the parsed json the subtree of "cars" objects and matches the resulting value with the extractor pattern JArrays(cars).
That is to say that the value is expected to be in the form of a constructor JArrays(something) and the something is bound to the cars variable name.

It works pretty much the same as you're probably familiar with case classes, like Options, e.g.

//define a value with a class that can pattern match
val option = Some(1)
//do the matching on val assignment
val Some(number) = option
//use the extracted binding as a variable
println(number)

The following assignments are exactly the same stuff

//pattern match on a JSon String whose inner value is assigned to "model"
val JString(model) = car / "model"
//pattern match on a JSon String whose inner value is assigned to "make"
val JString(make) = car / "make"

References

The JSON types (e.g. JValue, JString, JDouble) are defined as aliases within the net.liftweb.json object here.

The aliases in turn point to corresponding inner case classes within the net.liftweb.json.JsonAST object, found here

The case classes have an unapply method for free, which lets you do the pattern-matching as explained in the above answer.

can you explain what you mean unapply for free? Does this mean that all case classes will reverse the effect of the apply function if the case class is on the left side of the assignment operator? — jpswain, Jun 18 '13 at 02:20
It means that case classes generate a lot of boilerplate for you, and one of them is the unapply method that is used by pattern matching. And you can do pattern matching in the val statement (it will give a runtime error if it doesn't match). So you can do: case class C(a: Int, b: Int, c: Int); val x = C(1, 2, 3); val C(a, b, _) = x; assert(a == 1) — nafg, Jun 18 '13 at 06:09
+1 thanks I appreciate the explanation, it was very helpful to me. (And of course +1 and thanks for your original answer too!) — jpswain, Jun 18 '13 at 06:16

VasiliNovikov · Answer 3 · 2013-06-17T08:46:35.143

I think this should work for you:

case class UserInfo(
        name: String,
        firstName: Option[String],
        lastName: Option[String],
        smiles: Boolean
        )

val jValue: JValue
val extractedUserInfoClass: Option[UserInfo] = jValue.extractOpt[UserInfo]

val jsonArray: JArray
val listOfUserInfos: List[Option[UserInfo]] = jsonArray.arr.map(_.extractOpt[UserInfo])

I expect jValue to have smiles and name -- otherwise extracting will fail.

I don't expect jValue to necessarily have firstName and lastName -- so I write Option[T] in the case class.

In Scala what is the easiest way to parse json and map to objects?

3 Answers3

References