3

I have deployed neo4j community edition(version 4.0.0) in k8s and it has been exposed (bolt and browsers) through nginx-ingress and a load balancer.

Now when tried to connect to this db using py2neo(version latest), its working fine. But I got exception message: broken after 20 or 25 minutes of being idle.

Please be informed the idle-timeout time in AWS load balancer has been changed to the 3600s and also made this same for Nginx ingress.

exeption message: broken.

full log:

Traceback (most recent call last):
  File "/tmp/test/py2neo/wiring.py", line 270, in send
    n = self.__socket.send(self.__output)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
in query
    return self.graph.run(query).data()
  File "/tmp/test/py2neo/database.py", line 579, in run
    return self.auto().run(cypher, parameters, **kwparameters)
  File "/tmp/test/py2neo/database.py", line 918, in run
    result = self._connector.auto_run(self.graph.name, cypher, parameters, hydrant)
  File "/tmp/test/py2neo/client/__init__.py", line 996, in auto_run
    cx.sync(result)
  File "/tmp/tes/py2neo/client/bolt.py", line 318, in sync
    self._send()
  File "/tmp/test/py2neo/client/bolt.py", line 354, in _send
    sent = self._writer.send()
  File "/tmp/test/py2neo/client/packstream.py", line 652, in send
    return self._tx.send()
  File "/tmp/test/py2neo/wiring.py", line 273, in send
    raise WireError("Broken")
py2neo.wiring.WireError: Broken

The flow of the trafic is like this :

py2neo client ---> botl address(record in route53 against loadbalancer) ---> loadbalancer ---> nginx-ingress--->neo4j service---> neo4j docker container(standalone)

Can anyone please help me to solve this issue?

Taybur Rahman
  • 1,347
  • 8
  • 20

1 Answers1

1

What is your expectation here? Because py2neo contains no facilities to countermeasure hostile network environments in which connections are unexpectedly closed on a whim by third party network components.

Also, I don't understand your overall topology. You say you are using Community Edition, therefore you cannot be running a cluster, yet you mention a load balancer. What servers are you load balancing over? Community Edition can only run single instances. Also, why put nginx in there too?

To me, you have answered your own question. You have inserted network components that time out connections and py2neo is reporting exactly that. If you want to resume after such a connection failure, you will need to build client code to detect this and retry.

If you were using a casual cluster (and therefore Enterprise Edition) I would recommend the official Python driver instead. But I'm not sure that would help you much in this case.

Nigel Small
  • 4,475
  • 1
  • 17
  • 15
  • Thanks for your answer. Yes you are right, i have a standalone neo4j community edition DB running in k8s cluster(private topology). For exposing the neo4j service to the world we had to either use a loadbalancer or nodeport. We chose to use aws loadbalancer. We can also use the same loadbalancer to forward traffic based on url to multiple service using nginx-ingress controller. Which is why i have been using this loadbalancer and nginx-ingress to forward traffic to neo4j service which is running in k8s cluster. i have edited the question with traffic follow.Please check this. – Taybur Rahman Aug 26 '20 at 09:46
  • 1
    The clarity on topology is interesting, but it doesn't really change anything. The reason you are seeing this problem is because your third party network components are timing out the connections, and py2neo contains no facility to mitigate that. You have two choices: add error handling and retry logic to your application, or try the official driver instead, and see if the connection lifetime setting or similar helps you. – Nigel Small Aug 27 '20 at 12:58
  • Thanks for your Answer – Taybur Rahman Aug 27 '20 at 14:38