3

For example, in /proc/net/sockstat , does a TCP socket in CLOSE_WAIT get counted as 'inuse' or 'alloc' ?

In the kernel source net/ipv4/proc.c I see that sockstat_seq_show is called when getting the info from /proc/net/sockstat.

However I cannot see what differentiates a socket from being allocated (alloc) opposed to 'inuse'

[me@myhostname ~]$ cat /proc/net/sockstat
sockets: used 481
TCP: inuse 52 orphan 1 tw 66 alloc 62 mem 12
UDP: inuse 11 mem 5
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0

In net/tcp_states.h the possible states are enumerated as such

enum {
    TCP_ESTABLISHED = 1,
    TCP_SYN_SENT,
    TCP_SYN_RECV,
    TCP_FIN_WAIT1,
    TCP_FIN_WAIT2,
    TCP_TIME_WAIT,
    TCP_CLOSE,
    TCP_CLOSE_WAIT,
    TCP_LAST_ACK,
    TCP_LISTEN,
    TCP_CLOSING,    /* Now a valid state */
    TCP_NEW_SYN_RECV,

    TCP_MAX_STATES  /* Leave at the end! */

};

Which of the above count as 'inuse' and which count as 'alloc' ?

Party Time
  • 485
  • 5
  • 10

1 Answers1

5

Which of the above count as 'inuse' and which count as 'alloc' ?

You already got close to the answer by locating sockstat_seq_show - we can see that 'inuse' is the value of sock_prot_inuse_get(net, &tcp_prot), and 'alloc' is the value of proto_sockets_allocated_sum_positive(&tcp_prot). Now it's not always easy to follow the call chain further down, but I, if not mistaken, arrive at the following conclusions.

  1. 'alloc' - This at bottom is the sum of percpu_counter tcp_sockets_allocated, which gets incremented in tcp_init_sock(); there the socket state is initialized to TCP_CLOSE. Whatever state changes the socket undergoes during its existence, 'alloc' doesn't depend on - all TCP states count as 'alloc'.
  2. 'inuse' - This is the sum of the (per CPU) counters net->core.inuse or prot_inuse (for the TCP in this case), which essentially get incremented and decremented by calls of sock_prot_inuse_add(…, 1) resp. (…, -1) in inet_hash() resp. inet_unhash(). The condition in inet_hash() is if (sk->sk_state != TCP_CLOSE), so all TCP states except TCP_CLOSE count as 'inuse'.

I think this means in theory any socket in a state >= TCP_CLOSE is not counted as 'inuse'

In my view that can't be so, since also TCP_LISTEN > TCP_CLOSE, and a socket in TCP_LISTEN state surely is counted as 'inuse', as can be seen with e. g.

(cd /proc/net; cat sockstat; nc -l 8888& sleep 1; cat sockstat; kill $!; cat sockstat)|grep TCP
Armali
  • 18,255
  • 14
  • 57
  • 171
  • I think that just means sockets where sk->sk_state != TCP_CLOSE can increment the counter. inet_unhash() can still get called and decrement that counter. When we call inet_hash() and inet_unhash() functions we are working with the kernel's net connection hash table, which defines the following invariant (recall the enum of possible states) TCP_ESTABLISHED <= sk->sk_state < TCP_CLOSE which i believe means that inet_unhash() will be called when sk->sk_state > TCP_CLOSE which TCP_CLOSE_WAIT is. I think this means in theory any socket in a state >= TCP_CLOSE is not counted as 'inuse' – Party Time Mar 29 '19 at 17:45
  • Where did you derive the invariant from? - I expanded my answer. – Armali Apr 01 '19 at 06:27
  • 1
    You are correct, I tested on a server and sockets in CLOSE_WAIT are definitely counted as 'inuse' in sockstat. The invariant comes from the comments in inet_hashtables.h https://elixir.bootlin.com/linux/v4.8/source/include/net/inet_hashtables.h#L116 – Party Time Apr 09 '19 at 23:01
  • 1
    Very clear explanation, however from there I don't understand why we can see that tw > alloc here ? – Duarnad Apr 29 '21 at 07:48
  • I'm sorry - I've never seen this and have no explanation. – Armali Apr 29 '21 at 08:20
  • 1
    @Armali ok, thank you for having a look ! – Duarnad Apr 29 '21 at 08:26