What is the true maximum segment size of tcp in linux?

Question

I am able to gain the mss value from getsockopt:

tcpmss.c:

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
#include <stdio.h>

int main()
{
    int sockfd, mss;

    if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) < 0)
    {
        perror("sockfd");
        return 1;
    }

    socklen_t len = sizeof(mss);
    if (getsockopt(sockfd, IPPROTO_TCP, TCP_MAXSEG, &mss, &len) < 0)
    {
        perror("getsockopt");
        return 1;
    }

    printf("maximum segment size: %d\n", mss);
}

output:

maximum segment size: 536

other sources says, the default mss is 1460. But If I try to check it from client:

client.c:

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <netdb.h>
#include <string.h>
#include <unistd.h>

#define GET_CMD "GET %s HTTP/1.0\r\n\r\n"
#define SERV "80"
#define HOST "google.com"
#define HOMEPG "/"

//BUFSIZ = 8192, defined in <stdio.h>
int main()
{
    int sockfd, nbytes;
    struct addrinfo hints, *res;
    char buf[BUFSIZ];

    memset(&hints, 0, sizeof(hints));
    hints.ai_family = AF_UNSPEC;
    hints.ai_socktype = SOCK_STREAM;

    if (getaddrinfo(HOST, SERV, &hints, &res) != 0)
    {
        perror("getaddrinfo");
        return 1;
    }

    if ((sockfd = socket(res->ai_family, res->ai_socktype, res->ai_protocol)) < 0)
    {
        perror("socket");
        return 1;
    }

    if (connect(sockfd, res->ai_addr, res->ai_addrlen) < 0)
    {
        perror("connect");
        return 1;
    }

    nbytes = snprintf(buf, 256, GET_CMD, HOMEPG);
    if (write(sockfd, buf, nbytes) < 0)
    {
        perror("write");
        return 1;
    }

    while ((nbytes = read(sockfd, buf, BUFSIZ)) > 0)
    {
        printf("read %d bytes of home page of %s\n", nbytes, HOST);
    }
    if (nbytes == 0)
    {
        printf("got EOF from google.com");
    }
}

output:

read 8192 bytes of home page of google.com
read 3888 bytes of home page of google.com
read 7248 bytes of home page of google.com
read 4832 bytes of home page of google.com
read 6040 bytes of home page of google.com
read 6040 bytes of home page of google.com
read 6040 bytes of home page of google.com
read 4832 bytes of home page of google.com
read 2229 bytes of home page of google.com
got EOF from google.com

neither of those value is true. So I am little bit confuse with maximum segment size. I know the read() block and fetches more tcp segments into kernel receive buffer so I cannot see the true segment size from read() syscall, however, how to determine then the agreed window between the peers which should correspond to the MSS. Here on the first read(), I got full buffer, (BUFSIZE == 8192), then not even half, etc.

How to determine (all from my example)

1. MSS
2. propagated window between peers (and its relation to MSS)
3. how much segment size changes between each send operation (and why)

Answer 1

It's a relatively big question to answer since it includes many things.

Before getting into deep of answer, I think the most important thing to understand is that network is a complex and long path. We devide the path into serveral layers, and every layer may have differnt protocols which makes things more complex. So when you find a interesting thing about TCP, sometime we need to also look at the lower/upper layer to see the whole path.

Firstly, we always use tcpdump and wireshark to analysis network for more details, that can help you understand network deeply.

As for MSS, it is means Max Segment Size for TCP layer, since we use aonther limitation named as the maximum transmission unit (MTU) which is the size of the largest protocol data unit (PDU) that can be communicated in a single network layer transaction, MSS always need to cut down some little lengths for the header.

As for windows of TCP protocol, here are many different factors we must consider. We use a window because TCP need ACK to realise reliable transmission, the send/recv window can check for that(also develp for some other reasons). But with the bombing up of network stream, TCP add congestion control, so the window must edit by congestion window too. MSS/MTU is used as a factor to caclutate default value, but after that, many protocols/algorithms works together to make the window change for the reliable and efficient work of TCP connection.

For large packet, TCP can split them and then make them together when receiving. What's more, hardware can do it too. Here are many technologies like TSO（TCP Segmentation Offload), UFO（UDP Fragmentation Offload）, GSO（Generic Segmentation Offload）, LRO（Large Receive Offload）and GRO（Generic Receive Offload）.

So you see, it's really interesting and complex.