1

(the question is copied from msdn forum)

MSDN says this:

If you are using I/O completion ports, be aware that the order of calls made to WSASend is also the order in which the buffers are populated. WSASend should not be called on the same socket simultaneously from different threads, because it can result in an unpredictable buffer order.

Ok... but if I don't care about this order and also don't care about the order I will receive it on the other side... can I use it then? Is it safe to call WSASend from multiple threads? And will the packages be delivered how I pass them (maybe ordered differently, but complete) ??

E.g. -> I send [12a], [34b], [11c] and [223d] ... ([] marks the unit I pass to WSASend in one call) will I receive then e.g. 34b, 11c, 12a, 223d ? Or could it crash? Or could I receive things like 2a, 34b, 7a, 5c, ...

so, maybe the question should be -> is it thread safe? and is it atomic? (I'm talking about WSASend used with IOCPs, so OVERLAPPED)

maciekm
  • 257
  • 1
  • 5
  • 28
  • 1
    IIRC yes, you can do this. Such calls are thread-safe, but not TCP-safe:) – Martin James Sep 25 '14 at 20:06
  • Think about it - it's unusual, but not impossible, for two buffers to be returned by WSARecv() calls for the same socket and get handled by two different pool threads. Having processed the buffer data, those threads could possibly issue WSASend calls to send replies to the one socket 'at the same time'. So yes, they have to be thread-safe. – Martin James Sep 25 '14 at 20:12
  • My scenario aboce would be even more likely with a UDP protocol since the chances of two pool threads getting a complete datagram that demands a reply would be higher than for TCP. – Martin James Sep 25 '14 at 20:15
  • @Martin James thx for reply. What do you mean by TCP-safe ? In my server(tcp) there is a frequent situation, when many wsasends on the same socket from multiple threads are posted (one wsasend - one short message( up to 30 bytes), i dont care about buffer order, client side either). – maciekm Sep 25 '14 at 20:41

1 Answers1

4

It seems that actually calls to WSASend() are not thread safe and that if you intend to call WSASend() from multiple threads on the same connection then you should synchronise so that only one thread is actually calling into the API at any given point (i.e. hold a per connection lock around the WSASend() call). See this question and the attached test code for details: TCP/IP IOCP received data sometimes corrupt - Visual C++ on Windows and this blog entry for a more detailed explaination.

Also note that if you are sending distinct application level messages using multiple WSASend() calls from multiple threads then you have a problem as the multiple calls could be intermingled before the data gets written to the TCP stream.

So,

If you have 5 threads sending 'complete 2 byte messages' such that you send

AA, BB, CC, DD, EE etc. from 5 different threads then you will not get a stream that contains

ABCCDDBAEE

but you could get any combination of message order.

If your threads are sending messages with multiple send calls, such that three sends, A1 A2 A3 from thread 1 forms a single application level message. Then all bets are off as you could end up with

A1B1A2A3B2B3 etc. in the TCP stream.

If you need to do the later you will either need a lock of some kind around ALL of the WSASend calls so that you can group the multiple sends into a single application level atomic send or you should use multiple WSABUFs for a single WSASend call to gather together the multiple writes into a single call.

Community
  • 1
  • 1
Len Holgate
  • 21,282
  • 4
  • 45
  • 92