Is there a limit above max_dgram_qlen and wmem_{max,default}?

Question

I am trying to determinate the different limits of the unix datagram sockets, as I am using it as IPC for my project. The obscure thing I want to control is the size of my socket's internal buffer : I want to know how many datagrams I can send before my socket would block.

I've understood that 2 differents limits affect the size of the socket's buffer :

/proc/sys/net/core/wmem_{max, default} sets the max (-default) size of a socket's writing buffer
/proc/sys/net/unix/max_dgram_qlen sets the maximum number of datagram the buffer can hold

I know that /proc/sys/net/core/rmem_{max, default} sets the max (-default) size of a socket's reading buffer but as I am working on local unix socket it doesn't seem to have a impact.

I have set wmem_{max, default} to 136314880 (130 MB) and max_dgram_qlen to 500000. And wrote a small program where the sender socket only sends fixed size datagram to the receiver socket until is would block, I then print the size and number of datagram I was able to send.

Here is the code I used :

#include <err.h>
#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <sys/un.h>
#include <unistd.h>

/* Payload size in bytes. */
#define PAYLOAD_SIZE 100

#define CALL_AND_CHECK(syscall)                 \
    if (syscall < 0) { err(1, NULL); }

int main(void)
{
    int receiver_socket_fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_NONBLOCK, 0);
    if (receiver_socket_fd < 0)
        err(1, NULL);

    char* binding_path = "test_socket";
    struct sockaddr_un addr;
    memset(&addr, 0, sizeof(addr));
    addr.sun_family = AF_UNIX;
    strncpy(addr.sun_path, binding_path, sizeof(addr.sun_path));

    /* Check if the file exists, if yes delete it ! */
    if (access(binding_path, F_OK) != -1) {
        CALL_AND_CHECK(unlink(binding_path));
    }
    CALL_AND_CHECK(bind(receiver_socket_fd, (struct sockaddr const*) &addr, sizeof(addr)));

    int sender_socket_fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_NONBLOCK, 0);
    if (sender_socket_fd < 0)
        err(1, NULL);
    CALL_AND_CHECK(connect(sender_socket_fd, (struct sockaddr const*) &addr, sizeof(addr)));

    struct payload { char data[PAYLOAD_SIZE]; };
    /* Create test payload with null bytes. */
    struct payload test_payload;
    memset(&test_payload.data, 0, PAYLOAD_SIZE);

    ssize_t total_size_written = 0;
    ssize_t size_written = 0;
    do {
        size_written = write(sender_socket_fd, (const void *) &test_payload, PAYLOAD_SIZE);
        if (size_written > 0)
            total_size_written += size_written;
    } while (size_written > 0);

    printf("socket_test: %zu bytes (%ld datagrams) were written before blocking, last error was :\n", total_size_written, total_size_written / PAYLOAD_SIZE);
    perror(NULL);

    CALL_AND_CHECK(unlink(binding_path));
    CALL_AND_CHECK(close(sender_socket_fd));
    CALL_AND_CHECK(close(receiver_socket_fd));


    return 0;
}

I was expecting to reach either the max size in bytes of the socket (here 130MB) or the max number of datagram I set (500 000).

But the actual result is that I am only able to write 177494 datagrams before being blocked.

I can change the size of my payload it's always the same result (as long as I don't reach the maximum size in bytes first). So it seems that I am hitting a limit above max_dgram_qlen and wmem_{max, default} that I can't found.

I have of course tried to investigate ulimit or limits.conf without success. ulimit -b doesn't even work on my machine (says "options not found" and returns). I am working on Debian 10 (buster) but have launched my test program on different OS with the same result : I hit a limit of datagram that I don't know about.

Do you have any idea of which limit I didn't see and I am reaching ? And if I can read or modify this limit ?

Is there a limit above max_dgram_qlen and wmem_{max,default}?

0 Answers0