0

I am facing a problem with a simple function (language is C++) that iterates over a list of objects, serializes them and send them to a remote server. I am running this function on an ESP32 board that runs FreeRTOS and LwIP, so I have a TCP output buffer which is limited to 5744 bytes by default (I can customize it but I don't want to go over 10-15 thousand bytes).

Basically this is what happen: I call multiple times (in general 200-300 times) a write() or a send() on the socket, each time writing ~1400 bytes. After some time, if the server on the other side is a little bit slow, I saturate the TCP output buffer. Now, from my understanding, the send() or write() should wait until there's some space to write data again to the socket but instead the function immediately returns a value which is different than the value I wanted to write (let's say 1400 bytes).

char buffer[1460];
result = write(current_socket, buffer, 1460);
if(result != 1460){
    // error
} else {
    // everything ok
}

The socket is a default one, so it is a blocking socket and no options have been specified. I'd like the write to block until there's enough space to write all the 1460 bytes specified in the code above. The only reason the write should return an error should be a failure on the other side, for example the server closing the socket. Is this possible?

matteof93
  • 43
  • 9
  • This is correct an expected behavior of [write](http://man7.org/linux/man-pages/man2/write.2.html). You should just call it again until all data is written. – Piotr Praszmo May 15 '19 at 20:05
  • Here they say the send() should block https://stackoverflow.com/questions/14241235/what-happens-when-i-write-data-to-a-blocking-socket-faster-than-the-other-side but I noticed that send() and write() have the same behavior and do not block, they just return a value which is the amount of data they were able to write to the output buffer (0 or more but not the one that I expected). – matteof93 May 15 '19 at 20:16
  • @matteof93 that discussion talks about `send()`, but you are using `write()` instead. However, there is no guarantee that `send()` will block like they describe, that is implementation dependent. If there is any buffer space available, `send()` *can* return immediately with only a portion of the requested data buffered. You need to account for that possibility, like described in jch's answer. – Remy Lebeau May 15 '19 at 20:24

1 Answers1

2

What you're seeing is called a partial write. This is normal for TCP.

When the buffer is partly full, a call to write copies as much data as will fit, and returns the number of bytes copied into the buffer. It is your job to deal with the remainder of the data at a later time.

In order to send your buffer of data, you need to loop until the buffer is empty:

buf_ptr = 0;
buf_end = 1460;
while(buf_ptr < buf_end) {
    rc = write(fd, buf + buf_ptr, buf_end - buf_ptr);
    if(rc <= 0)
        break
    buf_ptr += rc;
}
jch
  • 5,382
  • 22
  • 41
  • Thank you. Will the server on the other side receive fragmented data or not? Because when I have to send let's say 1460 bytes I serialize them in Type-Length-Value format so the receiver can align with the sender simply reading the fixed length type field and then the fixed length field which will let it know how many bytes it should read for the value field. If I send my buffer with multiple calls to the write() function because of "partial write", will the receiver be able to run the same reading function it is used to run? – matteof93 May 15 '19 at 20:21
  • 1
    @matteof93 TCP is a stream, there is no 1-to-1 relationship between sends and reads, like there is in UDP. You are sending 1460 bytes, just keep looping as needed until all 1460 bytes have been sent. Yes, the receiver can still read your TLV format, it simply has to keep looping as needed until all of the expected Type, Length, and Value bytes have been received in full. – Remy Lebeau May 15 '19 at 20:25
  • Be aware that there's no guarantee that TLV boundaries will be aligned with `read` boundaries. – jch May 15 '19 at 20:34
  • I know that there's no guarantee about that but since I use 4 bytes for the type field and 2 bytes for the length field the first operation I do when I open the connection is to read 4 bytes (in a single read or multiple reads) and check if the type is valid, then if it's ok I go on with length which tells me how many bytes I'll have to read later fro the value. Doing this in a way similar to the code you posted above (I mean read until you read the bytes you need and they are what you expect) should allow me to send data correctly because I should be always aligned.... – matteof93 May 15 '19 at 20:43