5

I am using following code in Centos to change raw socket buffer size to 400 KB, however I got same result as I set buffer size to 256 KB. Anything wrong? or this is the limitation of socket layer? The kernel version is 2.6.34. Thanks!

int       rawsock;
socklen_t socklen;
int       optval;
int       bufsize = 400 * 1024;

rawsock = socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
if (rawsock < 0) {
    my_log(LOG_ERR, "error creating raw socket");
    return rawsock;
}

optval = 0;
socklen = 4;
err = getsockopt(rawsock, SOL_SOCKET, SO_RCVBUF, &optval, &socklen);
bail_error(err);
my_log("socket RX original buffer size = %d", optval);

optval = 0;
socklen = 4;
err = getsockopt(rawsock, SOL_SOCKET, SO_SNDBUF, &optval, &socklen);
bail_error(err);
my_log("socket TX original buffer size = %d", optval);

err = setsockopt(rawsock, SOL_SOCKET, SO_RCVBUF, &bufsize, sizeof(bufsize));
bail_error(err);

err = setsockopt(rawsock, SOL_SOCKET, SO_SNDBUF, &bufsize, sizeof(bufsize));
bail_error(err);

optval = 0;
socklen = 4;
err = getsockopt(rawsock, SOL_SOCKET, SO_RCVBUF, &optval, &socklen);
bail_error(err);
my_log("socket RX new buffer size = %d", optval);

optval = 0;
socklen = 4;
err = getsockopt(rawsock, SOL_SOCKET, SO_SNDBUF, &optval, &socklen);
bail_error(err);
my_log("socket TX new buffer size = %d", optval);

After running, the result is:

socket RX original buffer size = 110592
socket TX original buffer size = 110592
socket RX new buffer size = 524288
socket TX new buffer size = 524288
Yan x
  • 71
  • 1
  • 1
  • 6

3 Answers3

5

You're just hitting your system's current sysctl limits net.core.wmem_max and net.core.rmem_max.

If the process has superuser privileges, it can use the SO_SNDBUFFORCE and SO_RCVBUFFORCE ioctls to override the limits. If there is a real reason why your service does require larger buffers -- that is, any other reason besides poor development or design choices --, then I recommend this way. Usually there is no such reason, in which case I recommend you fix the application/service code instead.

You can modify the limits system-wide, but they'll affect all processes. Normally the defaults work fine, but in some specialized cases (embedded servers with very wide but long-latency network connections, perhaps?) you might wish to modify them.

To do this temporarily (until next boot), run sysctl -w net.core.wmem_max=bytes and sysctl -w net.core.rmem_max=bytes as root (where bytes is the new limit as a decimal number, in bytes).

To make the changes permanent, add

net.core.rmem_max=bytes
net.core.wmem_max=bytes

to your /etc/sysctl.conf file, or a new file in /etc/sysctl.d/ directory if your Linux distribution provides one. The latter is better approach, because it won't stop updates to your default configuration files.

If you want to delve deeper into these and other socket ioctls, you can take a look at the kernel net/core/sock.c file, and the sock_setsockopt() function therein.

Nominal Animal
  • 38,216
  • 5
  • 59
  • 86
2

According to the documentation

SO_RCVBUF : the maximum allowed value is set by the /proc/sys/net/core/rmem_max file

SO_SNDBUF : the maximum allowed value is set by the /proc/sys/net/core/wmem_max file.

So limits might depend on how your system has been configured.

dvhh
  • 4,724
  • 27
  • 33
0

Interesting side note. I coded an AF_ALG program that takes an input file (will be chosen randomly), encrypts it, decrypts it, then compares the decrypted plaintext against the original input (using cbc(aes)). The "test" program seemed to hang at 212992 bytes. Interrupt the program with ctrl-z, and put in background, and they reported reading 212992 bytes (which I know to be wrong for the test data, which was /bin/zip in this case, 215792 bytes).

Checked net.core.[rmem_max, wmem_max, rmem_default, wmem_default], and they were 212992. Updated to 2M, and tests that were previously failing started working. So it would seem that these limits also impact AF_ALG programs (I am trying to expose a crypto accelerator and put it through some stress from user-space using AF_ALG).

devoz
  • 31
  • 1