5

I have a nginx server with load balancing and reverse proxy. Right now its behing another nginx but very soon I plan to make it front, where it will receive TCP connections from clients directly at a rate of 500req/second

I am having some big troubles with the server. I have pasted my configurations here and I am kinda sure that the problem is with ipconntrac and similar things which are alient to me

http://paste.org/pastebin/view/28543

root@load_balancer:/proc/sys/net/ipv4# netstat -an|awk '/tcp/ {print $6}'|sort|uniq -c
     67 CLOSING
    727 ESTABLISHED
    173 FIN_WAIT1
    183 FIN_WAIT2
     19 LAST_ACK
      5 LISTEN
    447 SYN_RECV
      1 SYN_SENT
  27970 TIME_WAIT

Its a ubuntu machine with mainly nginx (load balancer and reverse proxy) installed.

It surely isnt great. Can you help me understand whats going on and how can I fix it. This is my live server and I am sure its in a bad shape right now. Any document or commands to fix this, or settings I should make to make this better and reduce time wait and fin_wait1/2 better would be awesome.

sysadmin1138
  • 133,124
  • 18
  • 176
  • 300
Sparsh Gupta
  • 1,127
  • 7
  • 21
  • 31

1 Answers1

9

Try the following:

echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse

Probably it will help to reduce the number of TIME_WAIT connections. You can also make these changes permanent in /etc/sysctl.conf:

net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_tw_recycle=1
Alex
  • 7,939
  • 6
  • 38
  • 52
  • 1
    +1 for a solution that works. However I don't know why, and as a developer, it hurts my head to understand this.... – Phung D. An May 10 '16 at 12:20
  • 1
    `tcp_tw_recycle` is not a good solution today. Basically it is removed in modern kernels - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4396e46187ca5070219b81773c4e65088dac50cc. `tcp_tw_reuse` can also cause issues behind load balancing devices - https://www.speedguide.net/articles/linux-tweaking-121. As a solution, check the max connection setting on your device and also look at possibility of enabling keep-alives – Aditya Nov 12 '18 at 22:31