Size of the data buffer in Unix Domain stream socket

Question

I have a client and a server implemented using Unix domain socket.

I want to see how much data the client can push on to the socket without the server reading off that data. I was expecting the client to be able to send around 20-30KB data before it got blocked. I have checked the net.core.rmem_default, net.core.wmem_default, net.core.rmem_max, net.core.wmem_max, and net.unix.max_dgram_qlen sysctl options and am sure i am not hitting these values. I also increased net.unix.max_dgram_qlen values but doesnt seem to help.

I was quite surprised to see that the number of fixed sized messages that i was able to send was around 138. This number remained constant even after i reduce the size of my messages.

On the client side i go into a loop and write 1024 messages.

CLIENT CODE

void write_text (int socket_fd, char* text)
{
    int length = strlen (text) + 1;

    send (socket_fd, &length, sizeof (length),0);
    /* Write the string. */
    send (socket_fd, text, length,0);
}

int main (int argc, char* const argv[])
{
   const char* const socket_name = argv[1];
   char message[100];
   int socket_fd;
   int loop = 0;
   struct sockaddr_un name;
   /* Create the socket. */
   socket_fd = socket (PF_LOCAL, SOCK_STREAM, 0);
   /* Store the server's name in the socket address. */
   name.sun_family = AF_UNIX;
   strcpy (name.sun_path, socket_name);
   /* Connect the socket. */
   connect (socket_fd, (struct sockaddr *) &name, SUN_LEN (&name));

   for (loop=0;loop<1024;loop++)
   {
      sprintf (message, "message number %d coming from the client", loop);
      /* Write the text on the command line to the socket. */
       write_text (socket_fd, message);
    }

    close (socket_fd);
    return 0;   
}

SERVER SIDE CODE:

unsigned int global_flag = 0;
int client_socket_fd = 0;

int server (int client_socket)
{    

  while (1) 
  {
    int length;
    char* text;

    if (read (client_socket, &length, sizeof (length)) == 0)
        return 0;

    text = (char*) malloc (length);

    read (client_socket, text, length);
    printf ("length %d %s\n", length, text);

    if (global_flag<5) break;

    free (text);
   }

    return 0;
}

int main (int argc, char* const argv[])
{
const char* const socket_name = argv[1];

int socket_fd;
struct sockaddr_un name;
int client_sent_quit_message;
socklen_t socket_length = sizeof(struct sockaddr_un);
int result;
int len = sizeof (int);
int data = 0;

socket_fd = socket (AF_LOCAL, SOCK_STREAM, 0);

name.sun_family = AF_UNIX;
strcpy (name.sun_path, socket_name);
socket_length = strlen(name.sun_path) + sizeof (name.sun_family);

bind (socket_fd, (struct sockaddr *) &name, socket_length);

listen (socket_fd, 5);

while (1)
{
        struct sockaddr_un client_name;
        socklen_t client_name_len;

        /* Accept a connection. */
        client_socket_fd = accept (socket_fd, (struct sockaddr *) &client_name, &client_name_len);

        client_sent_quit_message = server (client_socket_fd);            
}

/* Remove the socket file. */
close (socket_fd);
unlink (socket_name);
return 0;
}

In the server side code global_flag is always less than 0 and the server hence does one read and comes out. There are no more reads done by the server. The client meanwhile is pushing data on the socket.

I did a strace on the client and got this:

VirtualBox:~/code/linux$ strace ./unix-client /tmp/unixtest

execve("./unix-client", ["./unix-client", "/tmp/unixtest"], [/* 44 vars */]) = 0
brk(0)                                  = 0x1245000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =    0x7f3e12b63000
fstat(3, {st_mode=S_IFREG|0644, st_size=68001, ...}) = 0
mmap(NULL, 68001, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f3e12b52000
close(3)                                = 0

fstat(3, {st_mode=S_IFREG|0755, st_size=1815224, ...}) = 0
mmap(NULL, 3929304, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3e12583000
[clipped]
socket(PF_FILE, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_FILE, path="/tmp/unixtest"}, 15) = 0
sendto(3, "(\0\0\0", 4, 0, NULL, 0)     = 4
sendto(3, "message number 0 coming from the"..., 40, 0, NULL, 0) = 40
sendto(3, "(\0\0\0", 4, 0, NULL, 0)     = 4
sendto(3, "message number 1 coming from the"..., 40, 0, NULL, 0) = 40
sendto(3, "(\0\0\0", 4, 0, NULL, 0)     =  4
sendto(3, "message number 2 coming from the"..., 40, 0, NULL, 0) = 40
sendto(3, "(\0\0\0", 4, 0, NULL, 0)     = 4

..

sendto(3, "message number 138 coming from t"..., 42, 0, NULL, 0) = 42
sendto(3, "*\0\0\0", 4, 0, NULL, 0)     = 4
sendto(3, "message number 139 coming from t"..., 42, 0, NULL, 0) = 42
sendto(3, "*\0\0\0", 4, 0, NULL, 0

[AND THE CLIENT BLOCKS HERE]

Any idea why the client gets blocked after sending 139 messages when the server has stopped draining messages from the socket?

I would presume that the number of messages that the client can send on the socket would increase if i reduce the size of the message. However, i see that it remains constant. Irrespective of the size of the message the client is not able to send more than 139 messages on the socket without getting blocked.

Just a note, not actually relevant to your question: `sizeof` is an operator, not a function. The argument to `sizeof` should be a variable, an expression, or a parenthesized type name. So in the two places where you have `sizeof (length)`, it should really be either `sizeof length`, or `sizeof (int)`. — This isn't my real name, Aug 21 '14 at 01:08
The question is that why does the client get blocked after sending 139 messages irrespective of the size of the message. Where is this limitation coming from? — Toms, Aug 21 '14 at 04:20

score 2 · Answer 1 · answered Aug 21 '14 at 05:36

2

You're assuming that read() fills the buffer. It isn't specified to do that, only to transfer at least one byte. You need to store the return value in a variable, check it for -1, check it for zero, otherwise use it as the actual length of data received: if that is less than you were expecting, read again. Rinse and repeat.

answered Aug 21 '14 at 05:36

user207421

305,947
44
307
483

How does this go together with your comment regarding [Buffering characteristics of Unix Sockets](https://stackoverflow.com/a/12010348/2413201), EJP: _I don't see any reason why data that has been written wouldn't be available immediately on any platform._? – Armali Jun 01 '18 at 11:58
@Armali What I was addressing there was the claim that it is system-specific. What I am addressing here is the erroneous belief that `read()` fills the buffer. `read()` will block until at least one byte is available and then return whatever is available, even if it is less than the buffer size. There is no inconsistency. – user207421 Jan 10 '20 at 01:42

score 0 · Answer 2 · answered Jan 10 '20 at 09:24

As the question is about the blocking after a certain number of send() calls when There are no more reads done by the server, it's pointless to argue about read().
sourcejedi's answer to the question "What values may Linux use for the default unix socket buffer size?" explains that there's a fairly high overhead of e. g. 576 bytes per packet. While we don't use to think of packets in connection with a stream-oriented socket, we can derive an explanation for your observation if we assume that this overhead applies also to individual send calls on a UNIX domain socket of type SOCK_STREAM: A default buffer size of 163840 bytes divided by the sum of (4+576) bytes and about (41+576) bytes yields a number very close to the number of message pairs you were able to send.

Size of the data buffer in Unix Domain stream socket

2 Answers2