0

This is more of a design question while integrating couchbase with elastic search plugin. I have used couchbase with multitenancy in our previous product and we followed the the very first suggestion we found on couchbase blog here Single Couchbase Bucket for All Tenants.

Now we are researching on exploiting elastic search features on couchbase data using the couchbase elastic search plugin. Going through the couchbase elastic search plugin documentation (installing and setup) I realized that you will be able to map only one couchbase bucket to one elasticsearch index. That documentation can be found here Elasticsearch plugin configuration and here Connecting to Cluster. Now in that case, just as couchbase bucket, all the documents (regardless of the tenant) will reside in the same index.

Now here's my question. Regardless of how the documents are stored in couchbase, I would like elasticsearch to index to be per tenant. I am still quite new to playing around with the integration between these 2 systems but I am assuming that having separate search index per tenant (and each tenant/index having many different types of it's own) can most definitely 1) increase search performance per tenant 2) the performance of a particular search query on a specific tenant that might have minimal sets of data will not be impacted by having huge sets of data for some other tenant on the same index (although not plausible, assuming that the data sets between tenants differ by a factor of 50x)

What I am wondering is, are my concerns valid. Will performance on search queries be impacted by having all the tenants indexed together? And if so, anyone has any solutions on how can achieve this using couchbase elastic search plugin ?

All help and thoughts appreciated !

shahshi15
  • 2,772
  • 2
  • 20
  • 24

1 Answers1

1

There's no simple way to separate the data that comes from a single Couchbase bucket to different ElasticSearch indexes when using the plugin. What you CAN do is give the different tentants different types in ElasticSearch, while still keeping them in the same index. This will both improve the search performance, and make the tentant management consistent between CB and ES.

To map different documents from CB to different ES types, use the type selector feature of the transport plugin, as described here: http://docs.couchbase.com/admin/elastic/adv-usage.html (Scroll down to the section named "Setting document type")

David Ostrovsky
  • 2,461
  • 12
  • 13
  • But what if you have further types inside these tenants ? Should we at this point use combination types? For example, one of our types would be tenantId_users, tenantId_organizations ? Would that impact the performance as well ? – shahshi15 Feb 04 '15 at 15:15
  • Been playing around with the link you posted but it just ended up raising more questions. From the documentation, the types are basically hardcoded in the yml file and even if we do end up creating the types dynamically using elasticsearch native apis, how can we tell couchbase plugin to index based on the new type? I didn't find any way to actually dynamically create types using elasticsearch plugin. You can dynamically assign the documents to types based on the docId regex, but not create new types. Still doesn't solve the problem. – shahshi15 Feb 06 '15 at 00:31
  • 1
    The ability to create dynamic types (based on regex for both the ID and the type) was actually committed to the dev branch recently, and will be added in the next plugin version. So you will be able to create dynamic sub-types for the tentant documents. Also, consider whether the query performance problem you're trying to solve in the first place is a real one, or just premature optimization. – David Ostrovsky Feb 07 '15 at 09:49
  • That's great news. Is there any way we can get an earlier release to try it out ? – shahshi15 Feb 07 '15 at 20:26
  • You can get and build the latest version here: https://github.com/Branor/elasticsearch-transport-couchbase - remember to build it with the 1.6 JDK. Note that you'll need to build the CAPI server project first, as mentioned in the readme. This pull request describes how to use the regex type selector: https://github.com/Branor/elasticsearch-transport-couchbase/pull/1 – David Ostrovsky Feb 09 '15 at 17:14