Writing to neo4j pod takes much more time than writing to local neo4j

Question

I have a python code where I process some data, write neo4j queries and then commit these queries to neo4j. When I run the code on my local machine and write the output to local neo4j it doesn't take more than 15 minutes. However, when I run my code locally and write the output to noe4j pod in k8s pod it takes double the time, and when I build my code and deploy it to k8s and run that pod and write the output to neo4j pod it takes a round 3 hours. since I'm new to k8s deployment it might something in the pod configurations or settings, so I appreciate if I can get some hints

Answer 1

There could be few reasons of that.

I would first check how much resources does your pod consume while you are processing data, you can do that using kubectl top pod. Second I would check if there are any limits inside pod. You can read a great deal about them on Managing Compute Resources for Containers.

If you have a limit set then it might be too low and that's causing the extended time while processing data.

If limits are not set then it might be because of how you installed minik8s. I think as default it's being installed with 4G is memory, you can look at alternative methods of installing minik8s. With multipass you can specify more memory to allocate.

There also can be a issue with Page Cache Sizing, Heap Sizing or number of open files. Please read the Neo4j Performance Tuning.