0

I have a VPS server with 64 CPU cores and 128 GB RAM running on Debian. I have noticed that Google Bot from IP address 66.249.66.0/24 and Ahrefs Bot from IP addresses 54.36.148.0/24 and 54.36.149.0/24 are causing extreme CPU load up to 100%. This results in all websites on the server being very slow and almost unusable. There are many websites running on the server without caching.

Can you advise me on how to deal with this issue and improve the server's performance? Thank you!

Jakub Čuřík
  • 1
  • One option is to sign up as the webmaster / site owner in https://search.google.com/search-console and adjust the crawler frequency; other search engines might be adjusted via your robots.txt - but also everything that you can do to determine and optimize/fix the probably bad code that causes the load spikes will also benefit real site visitors – HBruijn Mar 27 '23 at 17:59

0 Answers0