ReactiveMongo + Play 2: stream a sub document (array)

Question

In the following example, I am using:

MongoDB (> 3.0 with WiredTiger engine)
Play framework 2.3.8 in Scala
The org.reactivemongo:play2-reactivemongo:0.11.0.play23-M3

First of all, suppose we have the following document in a MongoDB collection called for instance "urlColl":

{
  "_id" : ObjectId("5593bebe89645672000deec4"),
  "url" : "urlPath",
  "content" : [
      "a",
      "ab",
      "abc",
      "abcd"
  ]}

In Play, it is possible to stream new documents to the client side as soon as they are inserted in the "urlColl" collection, with the following method in Play:

def feed = Action {
  val cursor = collection
    .find(BSONDocument(), BSONDocument("content" -> 1))
    .options(QueryOpts().tailable.awaitData)
    .cursor[List[Try[BSONValue]]](ReadPreference.nearest)

  val dataProducer = cursor.enumerate(Int.MaxValue).map(_.toString)
  Ok.chunked(dataProducer &> EventSource()).as("text/event-stream")
}

and the implicit reader:

implicit object ContentToList extends BSONDocumentReader[List[Try[BSONValue]]] {
    def read(doc: BSONDocument): List[Try[BSONValue]] = {
      doc.getAs[BSONArray]("content").get.stream.toList
    }
}

Then, each time a new document with a "content" array is inserted, it is automatically sent to the client.

My question is quite simple:

Is it possible to stream to the client the new data injected in the "content" sub array?

Indeed, since there is no new document inserted (a new value is inserted in the array of the existing document), nothing is detected in the cursor and a fortiori nothing is sent to the client.

Thank you in advance!

@cchantep, to answer your last comment:

Yes, I am facing some slow queries. In fact, I have one form in a web page used to push data, and another web page where the server sends the events.

Validating the form of the first page can take up to 3 seconds. In the Mongo logs, I can see:

I QUERY    [conn329] getmore test.events cursorid:22982535122 ntoreturn:0 
keyUpdates:0 writeConflicts:0 numYields:0 nreturned:1 reslen:86 
locks:{ Global: { acquireCount: { r: 1000 } }, Database: { acquireCount: 
{ r: 1000 } }, Collection: { acquireCount: { r: 1000 } } } 2783ms.

This is a log of the events collection (2783ms) due to the tailable cursor. Maybe this can cause the problem?

Edit: I have disabled the tailable cursor (coll.find(selector).options(QueryOpts().tailable.awaitData)) and the performances are far better. Do you have any idea?

Thank you!

score 0 · Accepted Answer · answered Jul 01 '15 at 11:26

0

The final release of 0.11.0 is available as "org.reactivemongo" %% "play2-reactivemongo" % "0.11.0.play23".

About the sub-array, the tailable cursors in MongoDB (not only with ReactiveMongo) are monitoring new top-document. Since your document is updated to add an element into the array property, it has already been returned in the cursor stream when it has been created.

If you want to implement event source, it would be better to have tailable cursor over an event collection rather than on an entity collection (snapshot collection).

You can find a sample Play app using EventSource with Play Mongo Knockout.

answered Jul 01 '15 at 11:26

cchantep

9,118
3
30
41

Thank you @cchantep for your prompt answer! I am going to try your solution and will keep you updated. Moreover, do you have any information/links/opinion according the use of Akka as an alternative to tailable cursor for this use case? – Jul 01 '15 at 12:16
ReactiveMongo is using Akka under the hoods to implement reactive results, so if you can fit with event sourcing approach I don't think you will have to manage a custom actor system. – cchantep Jul 01 '15 at 14:50
Therefore, this is the best solution to stream documents from a MongoDB? It seems that the performance of such a solution is not (yet?) very good with a tailable cursor. Thanks! – Jul 01 '15 at 16:37
I have several deployed projects using tailable cursors. Do you encounter performance issue for now? – cchantep Jul 01 '15 at 17:44
I have made an answer to continue the discussion with more explanations. – Jul 01 '15 at 20:55

ReactiveMongo + Play 2: stream a sub document (array)

1 Answers1