Optimal way to read out JSON from MongoDB into a Scalatra API

Question

I have a pre-formatted JSON blob stored as a string in MongoDB as a field in one of collections. Currently in my Scalatra based API, I have a before filter that renders all of my responses with a JSON content type. An example of how I return the content looks like the following:

   get ("/boxscore", operation(getBoxscore)) {
        val game_id:Int = params.getOrElse("game_id", "3145").toInt
        val mongoColl = mongoDb.apply("boxscores")
        val q: DBObject = MongoDBObject("game_id" -> game_id)
        val res = mongoColl.findOne(q)
        res match {
            case Some(j) => JSON.parseFull(j("json_body").toString) 
            case None => NotFound("Requested document could not be found.")
        }
    }

Now this certainly does work. It doesn't seem the "Scala" way of doing things and I feel like this can be optimized. The worrisome part to me is when I add a caching layer and a cache does not hit that I am spending additional CPU time on re-parsing a String I already formatted as JSON in MongoDB:

JSON.parseFull(j("json_body").toString)

I have to take the result from findOne(), run .toString on it, then re-parse it into JSON afterwards. Is there a more optimal route? Since the JSON is already stored as a String in MongoDB, I'm guessing a serializer / case class isn't the right solution here. Of course I can just leave what's here - but I'd like to learn if there's a way that would be more Scala-like and CPU friendly going forward.

As JSON is just a string - can't you just return the string from your view? — Ross, Aug 23 '13 at 08:12

Stefan Ollinger · Answer 1 · 2013-08-25T20:44:00.280

There is the option to extend Scalatra's render pipeline with handling for MongoDB classes. The following two routes act as an example. They return a MongoCursor and a DBObject as result. We are going to convert those to a string.

get("/") {
  mongoColl.find
}

get("/:key/:value") {
  val q = MongoDBObject(params("key") -> params("value"))
  mongoColl.findOne(q) match {
    case Some(x) => x
    case None => halt(404)
  }
}

In order to handle the types we need to define a partial function which takes care of the conversion and sets the appropriate content type.

There are two cases, the first one handles a DBObject. The content type is set to "application/json" and the object is converted to a string by calling the toString method. The second case handles a MongoCursor. Since it implements TraversableOnce the map function can be used.

def renderMongo = {
  case dbo: DBObject =>
    contentType = "application/json"
    dbo.toString

  case xs: TraversableOnce[_] => // handles a MongoCursor, be aware of type erasure here
    contentType = "application/json"
    val ls = xs map (x => x.toString) mkString(",")
    "[" + ls + "]"

}: RenderPipeline

(Note the following type definition: type RenderPipeline = PartialFunction[Any, Any])

Now the method needs to get hooked in. After a HTTP call has been handled the result is forwarded to the render pipeline for further conversion. Custom handling can be added by overriding the renderPipeline method from ScalatraBase. With the following definition the renderMongo function is called first:

override protected def renderPipeline = renderMongo orElse super.renderPipeline

This is a basic approach to handle MongoDB types. There are other options as well, for example by making use of json4s-mongo.

Here is the previous code in a working sample project.

Optimal way to read out JSON from MongoDB into a Scalatra API

1 Answers1