Extract Options from potentially null JSON values using for expression

Question

I have a JSON document where some values can be null. Using for expressions in json4s, how I can yield None, instead of nothing?

The following will fail to yield when the value for either of the fields FormattedID or PlanEstimate is null.

val j: json4s.JValue = ...
for {
  JObject(list) <- j
  JField("FormattedID", JString(id)) <- list
  JField("PlanEstimate", JDouble(points)) <- list
} yield (id, points)

For example:

import org.json4s._
import org.json4s.jackson.JsonMethods._

scala> parse("""{
     |   "FormattedID" : "the id",
     |   "PlanEstimate" : null
     | }""")
res1: org.json4s.JValue = JObject(List((FormattedID,JString(the id)), 
    (PlanEstimate,JNull)))

scala> for {                                      
     | JObject(thing) <- res1                     
     | JField("FormattedID", JString(id)) <- thing
     | } yield id                                 
res2: List[String] = List(the id)

scala> for {                                      
     | JObject(thing) <- res1                     
     | JField("PlanEstimate", JDouble(points)) <- thing
     | } yield points
res3: List[Double] = List()
// Ideally res3 should be List[Option[Double]] = List(None)

score 3 · Answer 1 · answered Sep 08 '15 at 09:24

scala> object OptExtractors {
     |
     |   // Define a custom extractor for using in for-comprehension.
     |   // It returns Some[Option[Double]], instead of Option[Double].
     |   object JDoubleOpt {
     |     def unapply(e: Any) = e match {
     |       case d: JDouble => Some(JDouble.unapply(d))
     |       case _ => Some(None)
     |     }
     |   }
     | }
defined object OptExtractors

scala>

scala> val j = parse("""{
     |   "FormattedID" : "the id",
     |   "PlanEstimate" : null
     | }""")
j: org.json4s.JValue = JObject(List((FormattedID,JString(the id)), (PlanEstimate,JNull)))

scala>

scala> import OptExtractors._
import OptExtractors._

scala>

scala> for {
     |   JObject(list) <- j
     |   JField("FormattedID", JString(id)) <- list
     |   JField("PlanEstimate", JDoubleOpt(points)) <- list
     | } yield (id, points)
res1: List[(String, Option[Double])] = List((the id,None))

Is it somehow possible to deal in same manner with missed JSON fields? I.e. I want to get `None` if field is absent or field value is `null`. — WelcomeTo, Feb 19 '16 at 22:53
I'm afraid it's not going to work if there is a missing field. In that case, I think it would be better to avoid using for-comprehension. — thirstycrow, Feb 23 '16 at 15:20

Mik378 · Answer 2 · 2014-08-08T12:38:46.350

1

According to the documentation,

Any value can be optional. Field and value is completely removed when it doesn't have a value.

scala> val json = ("name" -> "joe") ~ ("age" -> (None: Option[Int]))

scala> compact(render(json))

res4: String = {"name":"joe"}

Explaining why your for comprehension doesn't yield anything.
Of course, a null value is mapped to None internally.

edited Aug 08 '14 at 12:38

answered Aug 08 '14 at 11:48

Mik378

21,881
15
82
180

The JSON is fetched & parsed at runtime. The json4s object contains a JNull for this field, so it's not removed - it's just treated as *empty* by the for expression. – Synesso Aug 08 '14 at 13:49

score 0 · Answer 3 · answered Sep 07 '15 at 11:45

The last command should look like:

for {
  JObject(thing) <- res1
} yield thing.collectFirst{case JField("PlanEstimate", JDouble(points)) => points}

Or like

for {
  JObject(thing) <- res1
  points = thing.collectFirst{case JField("PlanEstimate", JDouble(p)) => p}
} yield points

S.Karthik · Answer 4 · 2015-09-08T10:39:06.333

0

What about this

 for {
      JObject(thing) <- res1      
      x = thing.find(_._1 == "PlanEstimate").flatMap(_._2.toOption.map(_.values))
     } yield x

edited Sep 08 '15 at 10:39

answered Sep 08 '15 at 10:24

S.Karthik

1,389
9
21

Extract Options from potentially null JSON values using for expression

4 Answers4

Linked