1

For testing purposes I want to create a fully unbuffered file descriptor under linux in C. It shall have a reading and a writing side. It can be one of:

  • fifo
  • pipe
  • local or tcp socket
  • using stdin/stdout
  • virtual kernel file (e.g. /proc/uptime)

(I think the list is complete)

So a call to write() will copy the contents directly into the buffer provided by read().

Is this even possible or is there always a buffer in between? If it's possible than how to achieve this?

This is my code:

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>

int main(int argc, char** argv)
{
    int fd = socket(AF_INET, SOCK_STREAM, 0);
    int len = 0;
    setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &len, 4);
    struct sockaddr_in serv_addr = {}; 
    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(80); 
    inet_pton(AF_INET, "127.0.0.1", &serv_addr.sin_addr);
    connect(fd, (struct sockaddr *)&serv_addr, sizeof(serv_addr));
    char buf;
    read(fd, &buf, 1);
    return 0;
}

But the data is not read directly into buf. The reason is that my setsockopt() call does not really set the receiving buffer to zero. Instead it is set to the minimum value 256 as can be read on man page socket(7).

zomega
  • 1,538
  • 8
  • 26
  • Does this answer your question? [Using setvbuf() with STDIN stream](https://stackoverflow.com/questions/5876373/using-setvbuf-with-stdin-stream) – जलजनक Apr 30 '22 at 11:38
  • File descriptors do not have any buffering. Buffering is a part of `` API. – n. m. could be an AI Apr 30 '22 at 12:53
  • @जलजनक this is about settings the libc buffer. I want to set the kernel buffer to zero... – zomega May 27 '22 at 07:11
  • @n.1.8e9-where's-my-sharem. That's not true. Take a look at SO_SNDBUF and SO_RCVBUF for example. – zomega May 27 '22 at 07:54
  • SO_SNDBUF and SO_RCVBUF operate on the protocol level, they have nothing to do with your file descriptors. In particular, think what would happen if you were allowed to set SO_RCVBUF to zero. Data can arrive from the network at any time. Where would it be stored? – n. m. could be an AI May 27 '22 at 08:09
  • @n.1.8e9-where's-my-sharem. I added a code example... – zomega May 27 '22 at 08:42
  • `a call to write() will copy the contents directly into the buffer provided by read().` That is not possible. Use shared memory for that. The quesiton is too broad. Also I do not understand how to read unbuffered from /proc/uptime. – KamilCuk May 27 '22 at 08:44
  • @KamilCuk I think to proof that it's not possible to create a unbuffered fd I have to find out the minimum buffer size for all pipe/fifo/...? I thought there might be a call to set the buffer size when reading from /proc/uptime. But if there is no such call and there is a buffer in between then it's not possible to read this file unbuffered. – zomega May 27 '22 at 09:02
  • `call to set the buffer size when reading from /proc/uptime` This becomes _very_ specific depending on specific module. https://github.com/torvalds/linux/blob/master/fs/proc/uptime.c#L32 `%u` conversion uses a buffer by itself, I do not understand what exactly do you mean in case of such functions. – KamilCuk May 27 '22 at 09:06
  • `I have to find out the minimum buffer size for all pipe/fifo` Tradtionally, a pipe has a linked list of buffers each PIPE_BUF size. But there has been changes, I do not know how it works now, there are even ioctl calls. This is _very_ specific depending on what file descriptor refers to. And there is `splice()` and `sendfile()`. – KamilCuk May 27 '22 at 09:08
  • 2
    This smells of an XY problem. Why do you think you need to get rid of kernel buffers? – n. m. could be an AI May 27 '22 at 09:12
  • `man socket` SO_RCVBUF... The minimum (doubled) value for this option is 256. See https://elixir.bootlin.com/linux/latest/source/net/core/sock.c#L920 – n. m. could be an AI May 27 '22 at 09:28

1 Answers1

0

So a call to write() will copy the contents directly into the buffer provided by read()

Is this even possible

No. Only when using splice() or sendfile() or similar system call. This strongly depends on what the actual file descriptor refers to.

is there always a buffer in between? If it's possible than how to achieve this?

No. You can:

  • fifo / pipe
    • use splice() or sendfile()
    • use shared memory instead
  • local or tcp socket
    • open /proc/mem and write directly into the hardware.
    • write/use a specific kernel device driver that does the same
    • (I do not know, but maybe copy_from_user allows copying straight to a hardware address)
  • using stdin/stdout
    • file descriptor is just a pointer - both stdin and stdout can refer to a tcp socket, to a fifo, or to anything else
    • this depends on what the file descriptor refers to
  • virtual kernel file (e.g. /proc/uptime)
    • depends on the actual "virtual kernel file" implementation
    • most, if not all, implement seeking/support partial reading or use struct seq_file, which has to have an intermediary buffer.
KamilCuk
  • 120,984
  • 8
  • 59
  • 111
  • Actually there is a `vmsplice` system call that might be able to do it. It looks pretty tricky to use though. – Jan Hudec May 27 '22 at 09:42