Load elasticsearch data in knime

Question

I have a database loaded in elasticsearch and I would like to access it from KNIME to be able to make a recommendation system. How can I connect both applications?

As far as I know there's no connector in Knime for ES (yet), but you can use the [REST node](https://www.knime.com/forum/krest-nodes/get-request-for-elasticsearch-db) for that. Another connector worth looking into: https://www.cdata.com/kb/tech/elasticsearch-jdbc-knime.rst — Val, Feb 07 '18 at 11:38

score 1 · Answer 1 · answered Nov 05 '18 at 11:37

You can do it with python code in knime. Code is above

from pandas import DataFrame
output_table = DataFrame()

from elasticsearch import Elasticsearch
import json
import pandas as pd

esclient = Elasticsearch(['http://XX.YYYY.ZZZ.TTT:9200'])
response = esclient.search(
index='rtimlog-2018.11.02',
body={
   "query": {
    "match": {
      "message.eventParameters.Msisdn.keyword": {
        "query": "532XXXXX",
      }
    }
  }
}
)

columns=['A']
df = pd.DataFrame( columns=columns)


for hit in response['hits']['hits']: 
    X= json.dumps(hit['_source']['message'])
    df.loc[i]=X

output_table = df

score 1 · Answer 2 · answered Feb 02 '22 at 14:44

This is an older topic, and back then there were the REST, Python, or JDBC workarounds. I vaguely remember that the REST way was fiddly because KNIME does not support GET requests with body.

Since a few years there's fortunately now a dedicated commercial extension which supports reading from and writing to Elasticsearch (currently for version 6 and 7).

You can find more information here: https://nodepit.com/product/elasticsearch

Load elasticsearch data in knime

2 Answers2