-3

I want to send let's say 1GB data in 32 bits chunk so inside the for-loop I'm doing send(client_sock, buffer, tinfo->bufferSize, 0); Which is fine. But then after that, I'm receiving the response which is throwing an error(as below) after the 1st iteration. If I put the recv out side for loop then it works fine. But then I won't be able to ensure if each chunk is sent properly.

My question is

  1. Why can't we iterate over receive when we can iterate over
    send?
  2. Is it ok to keep receive out of the for and don't worry about if every message is sent?

N.B. - Can't use external libraries, its a POC project for college. 'send 1 Gb of data in 32 bits chunk over socket'

void *tcpClient(void *arg) {
    struct thread_info * tinfo = (struct thread_info *)arg;
    char * buffer = (char *)malloc(sizeof(char)*(tinfo->bufferSize));
    memset(buffer, 'a', sizeof(char)*(tinfo->bufferSize));

    int client_sock = socket(AF_INET, SOCK_STREAM, 0);
    connect(client_sock, (struct sockaddr *)&(tinfo->server_addr), sizeof(struct sockaddr));

    long noOfChunks = oneGb/ tinfo->bufferSize;
    for (int j = 0; j < noOfChunks; ++j) {
        int totalBytesRcvd = 0;
        int bytesRcvd = 0;
        send(client_sock, buffer, tinfo->bufferSize, 0);
        while (totalBytesRcvd < tinfo->bufferSize)
        {
            if ((bytesRcvd = recv(client_sock, buffer, tinfo->bufferSize, 0)) <= 0)
                error("recv() failed or connection closed prematurely");
            totalBytesRcvd += bytesRcvd; /* Keep tally of total bytes */

        }
        printf(buffer); /* Print the echo buffer */
    }

    free(buffer);
}

Error I'm getting:

Signal: SIGPIPE (signal SIGPIPE)
Terminated due to signal 13
user207421
  • 305,947
  • 44
  • 307
  • 483
sapy
  • 8,952
  • 7
  • 49
  • 60
  • C or C++, pick one please. Answers will be provided accordingly. – tadman Feb 28 '18 at 23:18
  • You can't jam 1GB of data into a socket without waiting for the socket to be writable, you'll flood the transmit buffer and probably crash your program. You'll need to use `select` to wait for the socket to be writable before attempting to write more data. If you want to ensure the data was transmitted correctly before writing more you'll need the client to acknowledge it somehow over the TCP stream. Most stream-based protocols have some kind of *framing* to deal with this. – tadman Feb 28 '18 at 23:19
  • 2
    "But then after that, I'm receiving the response which is throwing an error after the 1st iteration." - well what is the error?????? – user253751 Feb 28 '18 at 23:19
  • @tadman Yepp , so changed the program a bit to accommodate for message received completely . I'm just getting unspecified errir 'exit code 13' – sapy Feb 28 '18 at 23:24
  • @tadman, but looks like I can jam the socket , by iterating over `send` in a loop . Only problem is when I use `recv` it errors out. – sapy Feb 28 '18 at 23:26
  • Both send() and receive() have a return value. It can be -1 or 0, or anything upto (and including) their 3rd arguments. and, thus: `buffer[bytesRcvd] = '\0';` is wrong, and so is `send(client_sock, buffer, tinfo->bufferSize, 0);` – wildplasser Feb 28 '18 at 23:27
  • `send()` and `recv()` have a buffer behind them that is like 64KB, you can't just send 1GB of data like that... `send()` will return a number that tells you exactly how many bytes it managed to queue for sending in this buffer, if it's lower than the amount of bytes you tried to `send()` you have to manage that accordingly. – Havenard Feb 28 '18 at 23:38
  • @tadman You won't crash your program. You will *block.* – user207421 Feb 28 '18 at 23:42
  • @Havenard . Interesting, any pointer how to manage that . close connection , create new , and start from where left ? – sapy Feb 28 '18 at 23:42
  • Why can you not just send 1GB of data 'just like that'? If all the data cannot be buffered immediately, (as is likely the case!), then send() willl block until it has all been sent. – Martin James Feb 28 '18 at 23:48
  • I cannot see anywhere that the socket is set to non-blocking mode, so I don't understand many of the above comments - all that stuff about crashing. select etc:( – Martin James Feb 28 '18 at 23:50
  • Because I'm asked to measure the time difference when chunk size is changed, 32byte, 8 Byte etc. :( – sapy Feb 28 '18 at 23:50
  • 2
    What error are you getting? Use `perror()` so you see the reason why `recv()` failed. – Barmar Feb 28 '18 at 23:59
  • @MartinJames You are correct. It is all drivel. In blocking mode, `send()` will block until all the data has been transferred. It will not crash and it will not return a lesser count. – user207421 Feb 28 '18 at 23:59
  • I'm banging over my head for last 12 hours. If there is anything unclear please suggest , I'll edit the post . Please dont down vote (loosing points is ok, but I want an answer so that I can get up and grab something to eat :() – sapy Mar 01 '18 at 00:04
  • 2
    You've been told what's unclear, twice, and you've ignored it, twice. You will never get an answer until you tell us what the error was. It's impossible. – user207421 Mar 01 '18 at 00:14
  • /Users/diesel/Desktop/Assignments&Coursework/553/netcli/cmake-build-debug/netcli 32 1 Signal: SIGPIPE (signal SIGPIPE) Terminated due to signal 13 – sapy Mar 01 '18 at 00:31
  • 2
    @sapy please post this details in the question, not in the already very long comment section, people are going to overlook this detail. – Pablo Mar 01 '18 at 00:32
  • @sapy That buffer is internal to the system, you have to use the APIs provided by `socket.h` to manage sockets. Pretty sure there are functions to fetch how much space is available in the buffer (and change it's size aswell), but typically you just `select()` with a `writeset` to learn if there's space left and try to `send()` and check the return value of `send()` to know how much fit in there. – Havenard Mar 01 '18 at 00:55
  • Notice `send()` is always non-blocking for that reason, it doesn't really send anything, it just queues the data for sending because you don't have direct access to the network device. The system kernel will take care of sending it when the device isn't busy. – Havenard Mar 01 '18 at 00:57
  • @Havenard You are spreading a lot of misinfomation about this. `send()` is not 'always non-blocking`. It will block until all data has been transferred or an error occurs. This is clearly stated in the documentation. – user207421 Mar 01 '18 at 00:59
  • @EJP Sure, so if there's network latency `send()` will sit there for several seconds? This is unheard of. You would have to use multi-threading for almost everything. It is in fact double buffered, and I can even state that at least in Windows this buffer is 64KB by default (not sure about Linux). – Havenard Mar 01 '18 at 01:48
  • @EJP `send()` only blocks if you try to use it with the buffer full, not until stuff is transfered. – Havenard Mar 01 '18 at 01:51
  • @Havenard Blocks until all the data has been transferred into the buffer, just as it says in the documentation. If the buffer is full and there is latency, it will certainly block until there is room in the buffer. That could indeed take several seconds, or minutes, and it is not at all unheard of. NB 'Blocks' and 'always non-blocking' are irreconcilable, and you've claimed them both. Make up your mind. – user207421 Mar 01 '18 at 02:08
  • @EJP That's literally like saying everything is 'blocking' and there's no such thing as non-blocking because technically nothing happens instantly even if it happens really fast. Who's spreading misinformation here exactly? – Havenard Mar 01 '18 at 02:12
  • 1
    @Havenard 'You would have to use multi-threading for almost everything' is that a problem? You usually have two choices - multiple threads and blocking calls, or non-blocking async. If you use send() in default blocking mode, there is no issue with calling send() with large buffers. If you need to do other stuff while send() is blocked, then yes, you should use more than one thread. It's not unusual, strange or exceptional in any way. – Martin James Mar 01 '18 at 13:39
  • @MartinJames Those aren't the only options, you can use `select()` to wait for events on all sockets you are handling, simultaneously, so for all intents and purposes you are multi-tasking, without multi-threading, and nothing is blocking even though no socket is set to non-blocking. This is a really good way to handle things. Modern kernels support epoll that work even faster because you don't have to fill buffers with lists of FDs all the time, something that can be significantly resource consuming when handling thousands of connections. – Havenard Mar 01 '18 at 20:26
  • @MartinJames Personally I think non-blocking sockets should be avoided at all costs, trying to read from a socket when it's not ready is like being that annoying kid on the back seat that asks "Are we there yet?" every minute. This is far from professional and surely the worst way of handling multiple connections when you take performance in consideration. – Havenard Mar 01 '18 at 20:27
  • @Havenard I've seen 'non-blocking' clients with select(). Most seem to ignore the blocking 'connect' call at the start - a call that can take a while to complete:( – Martin James Mar 01 '18 at 20:31
  • 1
    @Havenard neither blocking sockets, nor non-blocking sockets with select(), require 'Are we there yet?' polling. – Martin James Mar 01 '18 at 20:34
  • @MartinJames Of course, but if you use non-blocking sockets it's because you have the intention to call a `recv()` or `send()` when the socket isn't ready, otherwise you wouldn't bother with it. I do agree with the `connect()` though, it can be handy to set it to non-blocking in that particular case. – Havenard Mar 01 '18 at 20:37
  • @Havenard More nonsense. Blocking mode sends block until all the data had been transferred. Try it before you debate this further. Saying so is not in the least 'literally like like saying that everything is blocking': and misrepresentation is not a valid form of argument. – user207421 Mar 01 '18 at 22:10
  • @EJP As far as I know there is no version of blocking where it waits until data is sent through the network. Blocking is only between the application and the kernel buffers. If there's space in the buffer to fit new data (when writing) or if there's data in the buffer to read, the respective `send()` / `recv()` returns immediately. `send()` will only block if the send buffer is full, and `recv()` will only block if the receive buffer is empty. – Havenard Mar 02 '18 at 05:07
  • @EJP A blocking `recv()` is indirectly waiting for a network event as obviously, when new data reaches the buffer it's because something came through, but it doesn't actually deal with networking directly, just as you are not required to have a `recv()` waiting for data at all times to be able to receive it. – Havenard Mar 02 '18 at 05:09
  • @Havenard Nobody has claimed that there is a 'version of blocking where it waits until data has been sent through the network' or that '`recv()` deal[s] directly with network events'. You are making this all up. We are discussing the behaviour of `send()` in blocking mode as defined by Posix. If you don't know what that is, look it up, but in any case stop these straw-man arguments. – user207421 Mar 08 '18 at 22:28
  • @EJP I have experience rewriting the whole socket library to validate IO in closed source applications that have security vulnerabilities. I'm pretty sure I know how the whole circus work. – Havenard Mar 08 '18 at 23:03

2 Answers2

2

Signal: SIGPIPE occurs when you write to a connection that has already been closed by the peer.

So, your peer has closed the connection.

Solution: don't.

Everything you've been told here about send() crashing the program or returning short counts in blocking mode is nonsense. Posix requires that it block until all the data is transferred or an error occurs, and this is exactly what is happening.

user207421
  • 305,947
  • 44
  • 307
  • 483
  • How to send 1 GB of data over socket then? – sapy Mar 01 '18 at 00:58
  • @sapy Just send it. That's not the problem. The problem is that the peer is closing its socket before receiving it all. – user207421 Mar 01 '18 at 00:59
  • @sapy wireshark it. When the peer closes the connection, you will see it. 'Debugging 101' says that you should split up complex problems until the are in solvable chunks. With a network issue, you first thought should be to use wireshark, or similar to identify which peer is causing the problem, so splitting the system into 'working' and 'not-working' client/server halves. If you don't do that, you may well end up in a black hole of fear, uncertainty and doubt:( – Martin James Mar 01 '18 at 13:47
0

1) @EJP's answer was almost correct but Let me rephrase it. "Socket receive can be in for loop", there is nothing wrong with it apart from it is inefficient and bad approach. And it you will probably get SIGPIPE error if Server closed connection while your loop is still active.

The obvious possible solution is to keep receive outside the loop as you don't need confirmation if each chunk sent (In most of the cases.)

2) It's ok to keep receive outside for is TCP is used, OS will take care if the chunks are sent in order and no chunks are lost. So even if we move receive outside the loop it's fine.

sapy
  • 8,952
  • 7
  • 49
  • 60
  • What do you mean 'almost'? And 'probably'? – user207421 Mar 08 '18 at 22:21
  • By almost I mean, Your answer never explicitly says, "Send and receive is possible inside the loop". "Probably " because there are times when the server does not close connection, and you don't get any error. – sapy Mar 09 '18 at 00:29