0

We have a Wagtail server deployed in one of the 3rd party datacenters. The deployment architecture is very simple and conventional. Proxy (Nginx)--> App Server --> DB. The portal looks good with medium traffic, but when some events happen in the university, there will be a peak load, and the proxy returns Busy Gateway (504).

What confusing us is the fact that the memory and resource utilization of each box is very low even when the proxy times out. We did some vertical scaling but the issue returns when the traffic peaks. proxy now runs on a 16 core machine, app runs on a 32 core machine and db runs on a 16 core machine.

Since we are not seeing any memory issue, we assume that Wagtail implementation is not the culprit. I know that the description given is very minimal but confused about how to start debugging this issue. Any pointers would be helpful. Thank you.

markwalker_
  • 12,078
  • 7
  • 62
  • 99
kallada
  • 1,829
  • 4
  • 33
  • 64
  • To really know what's going on in a situation like this you need performance monitoring from someone like https://newrelic.com/ They offer a fantastic service for really getting to know where your bottlenecks are. – markwalker_ Sep 12 '21 at 14:39
  • 1
    It could very well be the number of DB connections being requested at the same time. We experienced this with one of our apps, never got a 504, but we were hitting timeouts due to database requests. Try also looking into how many times you actually are hitting the database with the Wagtail code you have in place. Not an answer, but hope it will help eliminate some factors. – ja408 Sep 15 '21 at 07:22
  • @ja408 Thanks for the pointer and can you please give me some info nation on the solution for db connection issue – kallada Sep 15 '21 at 17:34
  • 1
    It really is a matter of refactoring your code if necessary. Make sure you aren't evaluating queries when they don't need to be. You can see some discussion of this in the docs here https://docs.djangoproject.com/en/3.2/topics/db/queries/#querysets-are-lazy. Another way to evaluate if it is the database at all is look at the database logs on your db server and watch them and of course you can run htop if you have a linux machine to watch performance. But 16 cores seems like you shouldn't have an issue. How much memory? htop on a linux machine would help determine that also. – ja408 Sep 15 '21 at 18:26

0 Answers0