1

I was wondering how many simultaneous sockets a linux box can open for TCP, UDP and ICMP communications in IPv4 and IPv6. While there are several replies about handling incoming (mainly TCP) connections, I didn't see a clear statement about outgoing.

My understanding is as follows:

  • TCP: as TCP is based on ports, one can only open a number of sockets equal to the number of ephemeral ports, which on my box is

    cat /proc/sys/net/ipv4/ip_local_port_range 
32768 60999 # == 28231

    If I want to increase that number (without changing the above configuration), I would need to create additional interfaces and I would gain about 28K additional connections per interface, right ?

    Is there any per system process consideration to be taken here ?

    I can see that working with additional (virtual) interfaces and public IPs, but will that still work if I define private IPs on virtual interfaces and NAT them through a unique public IP?

  • UDP: I believe the same holds for UDP as UDP also uses ports. If I have both UDP and TCP traffic, both would compete for the 28K ports, right ?

  • ICPM: ICMP is not based on ports so what is the limit on the number of open (raw) sockets ? Is it taken from the max number of opened FD pool:

    cat /proc/sys/fs/file-max
9223372036854775807

  • IPv6: all above was IPv4 related, but what are the differences between IPv4 and IPv6 on that matter ?

This is a bit messy as things are not very clear for me. Thanks in advance for your enlightenment!

mszmurlo
  • 1,250
  • 1
  • 13
  • 28

2 Answers2

2

... one can only open a number of sockets equal to the number of ephemeral ports

No. TCP connections must be unique regarding the set of source-ip, source-port, destination-ip and destination-port. Thus the limit regarding source-ports is only true if everything else is constant. This means for outgoing connections this limit applies only for connections to a specific fixed IP and port (assuming you have only a single IP on the outgoing interface).

And for UDP one has also to distinguish between a connected and unconnected socket. For connected sockets the same limits as for TCP connections apply. But there can be also unconnected UDP sockets which can sendto to arbitrary peers and also recvfrom from arbitrary peers, i.e. each sent or received packet can be unique regarding the peer. In this case the number of sockets is limited by the number of ports since each unconnected socket need (usually, see SO_REUSEPORT for exceptions) have a unique source-ip and source-port. But these sockets are on the other hand more flexible since a single socket can be used to communicate with arbitrary peers.

... what is the limit on the number of open (raw) sockets

There is no inherent limit. But it gets messy with too much raw sockets since all incoming data are delivered to all raw sockets.

IPv6: all above was IPv4 related, but what are the differences between IPv4 and IPv6 on that matter ?

No difference.

Apart from these limits there are of course other limits, like the number of open file descriptors per process, the number of file descriptors in total ... . But these limits can be tuned and are mostly hard-limited by the amount of memory available.

Steffen Ullrich
  • 114,247
  • 10
  • 131
  • 172
  • Hi. Does *(...) other limits, like the number of open file descriptors per process, (...)* this mean that if I have **one** process opening many, many sockets, the limit will be anyway about 65K ? – mszmurlo May 10 '21 at 16:17
  • @mszmurlo: *"... the limit will be anyway about 65K ..."* - No, why should it? – Steffen Ullrich May 10 '21 at 19:07
  • I'm probably mixing things, but isn't a socket a file descriptor and isn't the number of file descriptors per process limited to 0xFFFF (~ 65K)? – mszmurlo May 11 '21 at 04:31
  • @mszmurlo: *"but isn't a socket a file descriptor"* - yes. *"and isn't the number of file descriptors per process limited to 0xFFFF "* - I don't think so. Why do you think that this should be the case? – Steffen Ullrich May 11 '21 at 04:39
  • I don't know why, but I had in mind that the max number of open FD per process was limited to 65K. Thanks for your time. – mszmurlo May 11 '21 at 08:08
0

Well in this article

The Secret To 10 Million Concurrent Connections -The Kernel Is The Problem, Not The Solution

http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html

And the comments you find the description of a linux system that has 12 million is already possible. But pointing out that the 1970s (and 1990s design when using epoll - yes, over 20 years old now) is the main limit on current large system when we have 100GBit Ethernet cards.

Lothar
  • 12,537
  • 6
  • 72
  • 121