-1

For a while, my server has been returning 502 Bad Gateway errors randomly during the day. Sometimes we have the same traffic and everything is fine, other times it causes 502 errors. I have a web traffic of about 50r/s and here is the CPU and RAM usage attached.

I'm using PHP FPM 7.4 with config:

pm.max_childrens = 5
pm.start_servers = 2

VPS Specs:

8 vCore
16 GB Ram

CPU & Ram Usage: click here

PHP-FPM Error Log:

server reached pm.max_children setting (5), consider raising it

If I raise the value, for one minute everything works correctly and then after all the mysql requests return the following error: Lock wait timeout exceeded; try restarting transaction. So the site is unusable.

Thanks for your help

Elk Delf
  • 5
  • 4
  • Can you elaborate as to what's unclear to you about the error message? It's suggested you raise the `pm.max_children`, but your config seems to suggest you haven't done so. Why is that? – esqew Apr 11 '22 at 17:52
  • @esqew If I raise the value, for one minute everything works correctly and then after all the mysql requests return the following error: Lock wait timeout exceeded; try restarting transaction So the site is unusable – Elk Delf Apr 11 '22 at 18:08
  • Also, that behavior could be due to the PHP code we don't see. 502 is typical when your code has a fatal exception before it sent any header or html... The random factor could be due to some race conditions, or some code that is not always running etc. – Déjà vu Apr 11 '22 at 18:08
  • @Déjàvu All the php code works correctly, it's just the high traffic on the site – Elk Delf Apr 11 '22 at 18:10
  • That mysql lock error is usually happening when a deadlock occurs. So maybe your code is not totally "working correctly"? – Déjà vu Apr 11 '22 at 18:17
  • if you have high traffic, why do you think that keeping the max_children value low is a good idea? you should probably start with a *high* value, and gradually reduce it as you figure out what a good number is. – Kae Verens Apr 11 '22 at 18:17
  • @KaeVerens that's exactly what I did. 2048, 1024, 528, 200, 100 and 50. All the values above 50 only cause the mysql error 'Lock wait timeout exceeded; try restarting transaction'. The 502 errors are always present at 50. – Elk Delf Apr 11 '22 at 18:23
  • @Déjàvu Everything works when the traffic is low. – Elk Delf Apr 11 '22 at 18:24
  • Could you post last 100 lines of TEXT from your error log for analysis? – Wilson Hauck Apr 14 '22 at 14:18
  • When you post htop results, please also post complete TEXT output of SHOW FULL PROCESSLIST; so we can identify the ''running' MySQL processes. – Wilson Hauck Apr 14 '22 at 14:25

1 Answers1

1

You should consider raising pm.max_children and then think about MySQL optimization, start from configuration side, try to switch on slow_query_log to get a clue where might be a bottleneck. There is huge list of options and methods to extract more performance from MySQL server, but you need start from something and move forward step by step. Maybe show us your MySQL server configuration file and some data about load profile to give you better advice.