7

I have been using the Neo4j ElasticSearch integration module (https://github.com/neo4j-contrib/neo4j-elasticsearch) on the local, development and staging server given that the elasticsearch service is running on the server itself.

But when the Amazon AWS ElasticSearch service is used and data is added in the Neo4j db - sometimes the data is not getting inserted into elastic search.

There is no error or exception thrown while the transaction takes place between the Neo4j and Elasticsearch.

So it's getting difficult to debug as why is it happening.

Any ideas are appreciated.

Edit:

As for the network setup, the server is a EC2 instance where the Neo4j is installed and ElasticSearch service in question is "AWS ElasticSearch Service". As it does work sometimes I am not understanding any issue with the network here.

Neo4j version: 2.3.6 ElasticSearch version: 2.3.2

Again to point out, this only happens when the "AWS ElasticSearch Service" is connected and not with the one running on EC2 instance itself.

It would be of really great help to know if there is any way to log the transactions happening between the Neo4j and ElasticSearch services.

Below is the content of the log file: /var/log/neo4j/console.log

2016-09-02 12:27:47.494+0000 INFO  Remote interface ready and available at http://0.0.0.0:7474/
12:28:42.520 [NodeChecker RUNNING] ERROR i.s.c.config.discovery.NodeChecker - Error executing NodesInfo!
io.searchbox.client.config.exception.NoServerConfiguredException: No Server is assigned to client to connect
        at io.searchbox.client.AbstractJestClient$ServerPool.getNextServer(AbstractJestClient.java:132) ~[jest-common-2.0.2.jar:na]
        at io.searchbox.client.AbstractJestClient.getNextServer(AbstractJestClient.java:81) ~[jest-common-2.0.2.jar:na]
        at io.searchbox.client.http.JestHttpClient.prepareRequest(JestHttpClient.java:80) ~[jest-2.0.2.jar:na]
        at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:46) ~[jest-2.0.2.jar:na]
        at io.searchbox.client.config.discovery.NodeChecker.runOneIteration(NodeChecker.java:65) ~[jest-common-2.0.2.jar:na]
        at com.google.common.util.concurrent.AbstractScheduledService$ServiceDelegate$Task.run(AbstractScheduledService.java:189) [guava-19.0.jar:na]
        at com.google.common.util.concurrent.Callables$3.run(Callables.java:100) [guava-19.0.jar:na]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_101]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_101]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_101]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
12:28:42.540 [NodeChecker RUNNING] INFO  i.s.client.AbstractJestClient - Setting server pool to a list of 1 servers: [ELASTICSEARCH_URL]
12:29:42.541 [NodeChecker RUNNING] DEBUG i.s.client.http.JestHttpClient - GET method created based on client request
12:29:42.541 [NodeChecker RUNNING] DEBUG i.s.client.http.JestHttpClient - Request method=GET url=ELASTICSEARCH_URL/_nodes/_all/http
12:29:42.553 [NodeChecker RUNNING] DEBUG io.searchbox.action.AbstractAction - Request and operation succeeded
12:29:42.553 [NodeChecker RUNNING] DEBUG i.s.c.config.discovery.NodeChecker - Discovered 0 HTTP hosts:
12:29:42.553 [NodeChecker RUNNING] INFO  i.s.client.AbstractJestClient - Setting server pool to a list of 0 servers: []
12:29:42.553 [NodeChecker RUNNING] WARN  i.s.client.AbstractJestClient - No servers are currently available to connect.

The response from the API: ELASTICSEARCH_URL/_nodes/_all/http

EC2 instance:

{"cluster_name":"elasticsearch","nodes":{"X9zagEOlSK-h3l9dSG08PA":{"name":"Her","transport_address":"172.31.50.210:9300","host":"172.31.50.210","ip":"172.31.50.210","version":"2.3.0","build":"8371be8","http_address":"172.31.50.210:9200","http":{"bound_address":["[::]:9200"],"publish_address":"172.31.50.210:9200","max_content_length_in_bytes":104857600}}}}

AWS ElasticSearch instance:

{"cluster_name":"102372860153:ES_DONAIN_NAME","nodes":{"kXO7l2ZyRgaDq44Ohx4qCA":{"name":"Cassie Lang","version":"2.3.2","build":"0944b4b"}}}
alexwlchan
  • 5,699
  • 7
  • 38
  • 49
Kunal Dethe
  • 1,254
  • 1
  • 18
  • 38
  • I think you are going to have to give us some code. From what you have told us there is nothing to go on. – Daniel Wisehart Sep 12 '16 at 13:58
  • Hello @DanielWisehart, I have updated the question by providing the logs. Does this help in any way? And for the code thing - the integration module that I have mentioned says to add the mapping in the configuration file like `elasticsearch.host_name=http://localhost:9200 elasticsearch.index_spec=people:Person(first_name,last_name), places:Place(name)` so that when the same data in entered into Neo4j using any connection adapter, the sync will be handled by the plugin (.jar) files added to Neo4j plugins folder. There is no code as such in this. – Kunal Dethe Sep 13 '16 at 08:11
  • That helps, Kunal, but I still don't have enough to run a test case for myself and I haven't seen this console.log exception before. You say the exception and warning are not happening during the transaction so you think they are not related to your problem? Can you give us enough to recreate the problem for ourselves? – Daniel Wisehart Sep 13 '16 at 11:51
  • By "enough" I mean the configuration and sample data to recreate the problem. You know, the usual MCVE: http://stackoverflow.com/help/mcve – Daniel Wisehart Sep 13 '16 at 11:58

0 Answers0