0

I'm running crawler on my company's internet. 10 raspberry pi * 45 crawlers each, 2 desktops * 70 crawlers each

These processes are sending requests 24/7.

3~5% of packets are getting lost. This is affecting queuing system heavily.

I'm using my internet company's wifi router and Ubiquiti Unifi 24-port PoE switch.

The router has a quad-core CPU and 512 MB RAM.

Can these many requests cause packet loss?

startergate
  • 1
  • 2
  • if i understood your question correctly, then my answer is: if all the requests are getting NAT to a single IP and are sent to your ISP, then this might be a cause. since ISPs can limit concurrent sessions from a Single IP toward their services. – Zareh Kasparian Mar 11 '22 at 12:52
  • @ZarehKasparian that could be my problem! I will try multi wan routers. – startergate Mar 12 '22 at 04:40

0 Answers0