recv with MSG_NONBLOCK and MSG_WAITALL

Question

I want to use recv syscall with nonblocking flags MSG_NONBLOCK. But with this flag syscall can return before full request is satisfied. So,

can I add MSG_WAITALL flag? Will it be nonblocking?
or how should I rewrite blocking recv into the loop with nonblocking recv

Guess you want to spare user-process memory (to buffer the incomplete message), so you wish to use kernel-memory. I doubt it would work. — Lorinczy Zsigmond, Feb 17 '18 at 20:42

score 5 · Answer 1 · answered Jun 11 '13 at 15:23

For IPv4 TCP receives on Linux at least, MSG_WAITALL is ignored if MSG_NONBLOCK is specified (or the file descriptor is set to non-blocking).

From tcp_recvmsg() in net/ipv4/tcp.c in the Linux kernel:

if (copied >= target && !sk->sk_backlog.tail)
        break;

if (copied) {
        if (sk->sk_err ||
            sk->sk_state == TCP_CLOSE ||
            (sk->sk_shutdown & RCV_SHUTDOWN) ||
            !timeo ||
            signal_pending(current))
                break;

target in this cast is set to to the requested size if MSG_DONTWAIT is specified or some smaller value (at least 1) if not. The function will complete if:

Enough bytes have been copied
There's a socket error
The socket has been closed or shutdown
timeo is 0 (socket is set to non-blocking)
There's a signal pending for the process

To me this seems like it may be a bug in Linux, but either way it won't work the way you want. It looks like dec-vt100's solution will, but there is a race condition if you try to receive from the same socket in more than one process or thread.
That is, another recv() call by another thread/process could occur after your thread has performed a peek, causing your thread to block on the second recv().

Surely `timeo` being zero just means there is no read timeout set? — user207421, Feb 06 '20 at 03:21

Duck · Answer 2 · 2010-05-31T16:39:58.950

EDIT:

Plain recv() will return whatever is in the tcp buffer at the time of the call up to the requested number of bytes. MSG_DONTWAIT just avoids blocking if there is no data at all ready to be read on the socket. MSG_WAITALL requests blocking until the entire number of bytes requested can be read. So you won't get "all or none" behavior. At best you should get EAGAIN if no data is present and block until the full message is available otherwise.

You might be able to fashion something out of MSG_PEEK or ioctl() with a FIONREAD (if your system supports it) that effectively behaves like you want but I am unaware how you can accomplish your goal just using the recv() flags.

NONBLOCK can return with only part of required message. I want to get EAGAIN form non-blocking recv, if it want to return only part of msg. So, I want nonblocking recv with "all or nothing" behaviour — osgx, May 31 '10 at 13:16

score 3 · Answer 3 · answered Jun 07 '11 at 21:10

This is what I did for the same problem, but I'd like some confirmation that this works as expected...

ssize_t recv_allOrNothing(int socket_id, void *buffer, size_t buffer_len, bool block = false)
{
    if(!block)
    {
        ssize_t bytes_received = recv(socket_id, buffer, buffer_len, MSG_DONTWAIT | MSG_PEEK);

        if (bytes_received == -1)
            return -1;

        if ((size_t)bytes_received != buffer_len)
            return 0;
    }

    return recv(socket_id, buffer, buffer_len, MSG_WAITALL);
}

recv with MSG_NONBLOCK and MSG_WAITALL

3 Answers3