1

I am asking some theoretical questions, since it would be really hard for me to post the code of the project involved, which is composed of too many files.

I am writing code for a server program, which has to communicate with several clients that send a variety of different requests and expect answers for each of them. The server is multi-threaded, so every single thread mutually accesses a client connections list and performs all the operations within the request.

The two parts are communicating viaAF_UNIX sockets and for mutual exclusion I have used locks and condition variables.

Now my issue is this: with certain interleavings of execution, the server side ends up making two simultaneous writes to the client (actually two worker threads within the server both send a message to the same client), which is expecting just one. For unlucky interleavings, the client only gets to read one of the messages sent by the server, but I have noticed that sometimes this doesn't happen and everything works fine, this is because the two different requests to the client happen to be distanced in time.

The issue happens even when one server thread does a write before the client calls read, and in between those two events, another server thread calls another write to the same client. In this case, only the most recent write is received by the client.

From what I have understood regarding blocking mode (which is what I am using)read() and write() should block when no one's receiving from the other side. Now I don't understand why the second write from the server worker gets completely lost? Shouldn't it block if no one's receiving and then resume when the client calls read?

Should I usemutual exclusion on the socket so that the second write should wait for the previous one to be completely finished?

I hope that my issue is clear even though I am not showing any code, if necessary please tell me and I will try to post some pieces of code. I think that the issue might just be a conceptual thing, regarding my understood of read(), write() and mutual exclusion, but I understand that the issue might be somewhere else and that without code it would be hard to figure out! Thank you!

Harini
  • 551
  • 5
  • 18
BattiestFawn66
  • 67
  • 3
  • 10
  • `regarding blocking mode (which is what I am using)read() and write() should block when no one's receiving from the other side.` Wrong. They block when there is nothing to read(yet) or when the local buffersare full and the write is throttled. If the *other end*of the connection crashes or exits, the connection breaks resulting in EOF (or EPIPE, or CONNECTION_RESET, etc) on your side. – wildplasser Jun 17 '17 at 18:46
  • ... assuming TCP... UDP is different. (except forthe EOF/broken connection -thing, which is *completely* different) – wildplasser Jun 18 '17 at 01:24
  • Do you check the `send()`/`write()` return values, or do you simply assume they succeed? Not checking them would explain the lost data; otherwise the lost data is a mystery. – Nominal Animal Jun 18 '17 at 01:56
  • @NominalAnimal yes I do check both return values and read/write bytes ! – BattiestFawn66 Jun 18 '17 at 10:24
  • @wildplasser I am not using TCP on this project but I think your comment applies to my situation as well... I went on analyzing the problem I think the issue is not much with the blocking concept, but with the fact that the client does one read of a message( the message size is fixed) , while the server makes two writes... do you think putting a mutex on the writes would work? – BattiestFawn66 Jun 18 '17 at 10:24
  • @BattiestFawn66: What is the message size? – Nominal Animal Jun 19 '17 at 02:03
  • @BattiestFawn66: I cannot reproduce datagram loss using Unix domain datagram socket pair, using Linux kernel 4.4.0 on x86_64. (I've a test program that uses a number of threads sending on the same file descriptor, single receiver thread, verifying the contents and sequence of each message.) With an Unix domain stream socket pair, the contents are usually interleaved, of course. Are you using a datagram socket? (You definitely should be.) – Nominal Animal Jun 19 '17 at 03:45

0 Answers0