0

I wanted to know if the DCP based streaming is stable in Spark Couchbase Official Connector? The documents state that they aren't ready to be used in production as yet. Do we have any examples for Java?

Also, if someone could shed some light on the use case that I am trying to solve and whether a spark connector makes sense here:

We get daily updates about some badges from a couchbase lite db to sync gateway. Now, over a window of 10 days, if someone earns 5 or more badges, then we need to push a congratulatory badge. This needs to be done after every 10 days.

So, what I was planning was that using the DCP, I check for changes on a daily basis, run a job and then push it to sync gateway. If within the 10 day interval, 5 or more badges have been earned then I push it to the sync gateway..

Is this the right way to go about this or are there better ways to do this?

One more thing that could be used is:

http://developer.couchbase.com/documentation/mobile/1.2/develop/guides/sync-gateway/server-integration/changes-worker/index.html..

So which approach would be better of here?

Sagar
  • 229
  • 4
  • 14

1 Answers1

0

The DCP streaming from the Spark Connector is supported as that product is GA and that feature is not marked experimental. Couchbase does introduce features early and marks them experimental through annotations if the feature is not supported.

That said, I know there are some known issues with the JVM Core DCP implementation as of this writing which are expected to be fixed in the next couple weeks, if not sooner. The one I know of for sure is tracked here: https://issues.couchbase.com/browse/JVMCBC-312

New test scenarios have been added and Couchbase is working on making sure it passes these tests during different failure/rebalance modes before the next release.

Matt Ingenthron
  • 1,894
  • 14
  • 14
  • Thank you for the response.. If it were to be used for reading streams from the changes feed from the sync gateway, does it keep track of the lastSequnce number i.e till what point has it read? OR does it always read all of the data from the bucket? – Sagar Mar 22 '16 at 05:41
  • The question there doesn't quite apply, since last sequence number is not a concept at the Couchbase bucket level, only at the Couchbase Sync Gateway level. The Spark Connector uses DCP directly, not Sync Gateway which abstracts away some cluster concepts and adds some more concepts. See more in the docs here: http://developer.couchbase.com/documentation/mobile/current/develop/guides/sync-gateway/running-sync-gateway/connecting-sync-gateway-to-couchbase-server/index.html – Matt Ingenthron Mar 28 '16 at 23:32