Elasticsearch document source in carrot2

Question

Just wondering if the Carrot2 developers plan on creating a source for elasticsearch? i.e. org.carrot2.source.elasticsearch

I saw the plugin directly within elasticsearch, but in my case I would like to be able to use the Carrot2 webapp.

Answer 1

There are two problems with writing a document source for ES:

• we don't know what schema the user will define for their data (ES is schemaless),
• the queries issued to ES can be complex JSON and are typically tightly bound to a particular schema.

Given the above it's difficult to write a document source that would be generic enough to satisfy all the potential scenarios. It could be something that assumes "hardcoded defaults" but I'm not really convinced it's the right way to go.

Ideally, you should consider writing your own implementation of IDocumentSource that will know the schema (and details) of your ES setup to formulate the request and parse the output.