1

As per cppreference,

When an evaluation of an expression writes to a memory location and another evaluation reads or modifies the same memory location, the expressions are said to conflict. A program that has two conflicting evaluations has a data race unless:

a) both evaluations execute on the same thread or in the same signal handler, or
b) both conflicting evaluations are atomic operations (see std::atomic), or
c) one of the conflicting evaluations happens-before another (see std::memory_order)

I am confused a little with respect to point c. As per Anthony Williams's book(C++ concurrency in action), section 5.1.2:

If there’s no enforced ordering between two accesses to a single memory location from separate threads, one or both of those accesses is not atomic, and if one or both is a write, then this is a data race and causes undefined behavior. According to the language standard, once an application contains any undefined behavior, all bets are off; the behavior of the complete application is now undefined, and it may do anything at all

I understand that if I enforce the ordering, say by using std::mutex, so that it's not possible for more than one evaluation(of the conflicting expression) to happen at the exact same time as another, then everything is fine and my program will be well defined. I like to think of this as 'compile-time order enforcement'.

I wanted to understand if 'run-time order enforcement' is sufficient to eliminate data-race/undefined behaviour.

Say for example, I design a client-server system like below:

Server Specifications

  • It will be using 2 threads(Thread A and Thread B).

  • Both threads will be sharing a global int variable initialized with 0.

  • It will be listening for client's messages on 2 ports(port A and port B).

  • Thread A will use port A and Thread B will use port B.

  • Thread A/B pseudo code:

      while(true) {
          Receive message on the connected socket(port A in case of Thread A and port B for thread B).
    
          Increment the shared global variable.
    
          Send an acknowledgement to the client.
      }
    
  • Please note that, there's no inter-thread synchronisation introduced by the networking library/system calls made in the first and last step of the pseudo code.

Client Specifications

  • It will be single threaded

  • It will connect with the above mentioned 2 ports of the server.

  • It will alternately send messages to both the ports of the server, starting with port A.

  • It will send the next message only after receiving the acknowledgment of the previous message.

  • Pseudo code:

      while(true) {
          Send message to port A of server.
    
          Receive acknowledgement of the above message.
    
          Send message to port B of server.
    
          Receive acknowledgement of the above message.
      }
    

In the above example, server's Thread A and Thread B are sharing a global variable. There are 2 conflicting expressions(increment operation of shared variable) and I haven't used any language provided synchronisation mechanism. However, I enforce the ordering at run-time. I understand that the compiler cannot know about the runtime but on run time, because I ensure it, both thread A and thread B can never access the variable at the same time.

So I'm not sure if this falls in the category of data race.

So basically my queries are:

  1. Is it necessary to enforce ordering at compile time rather than run time to avoid data race? Will the above server's code fall in the category of programs having data race?

  2. Is there a way to enforce ordering at compile time in a multi-threaded C++ program when threads are sharing data without using C++ language's synchronisation constructs(mutex/futures/atomic) etc.

Vishal Sharma
  • 1,670
  • 20
  • 55
  • It might so happen that "Receive message on the connected socket" internally synchronises on some shared state, which will give a happens-before relationship to your code (that you pedagogically didn't want) – Caleth Aug 12 '21 at 09:57
  • @Caleth is the design/way of running the system as mentioned in the question(sending next message only after receiving ack) not enough to guarantee a 'happens before relation'? I suspect it's not but I don't understand why? I mean the threads will literally work on the shared variable only when they'll receive the message and we are making sure that both don't receive at the same time. – Vishal Sharma Aug 12 '21 at 10:06
  • 1
    Yes, happens-before is not about a wall-clock-time of when things happen – Caleth Aug 12 '21 at 10:14
  • I think I see what you mean. After going through, the points given in 'happens-before' section of https://en.cppreference.com/w/cpp/atomic/memory_order, I can see that artificially trying to enforce order via client(in above scenario) falls in none of the cases mentioned in that link. – Vishal Sharma Aug 12 '21 at 10:48

2 Answers2

3

Will the above server's code fall in the category of programs having data race?

Yes.

Is there a way to enforce ordering at compile time in a multi-threaded C++ program when threads are sharing data without using C++ language's synchronisation constructs(mutex/futures/atomic) etc.

Implementations can provide additional guarantees beyond what the standard requires.

Caleth
  • 52,200
  • 2
  • 44
  • 75
  • So will it be correct to say that run time order enforcement is not sufficient? It has to be at compile time(by using synchronisation primitives given by the language/OS). – Vishal Sharma Aug 12 '21 at 09:58
  • Wouldn't sufficient waiting invoke the Forward Progress guarantees and make the changes to the shared variable visible? – dyp Aug 12 '21 at 10:48
  • @dyp we are assuming that the networking code doesn't internally synchronise these threads (because otherwise it wouldn't be the demonstrative example we want). – Caleth Aug 12 '21 at 11:06
  • @Caleth do you know if this same program above has a data race as per C memory model as well? – Vishal Sharma Aug 20 '21 at 05:05
0

Will the above server's code fall in the category of programs having data race?

No if you wait for acknowledgement. But you are walking on very thin ice. What if some time later, you send both requests together or maybe even one after other without waiting for acknowledgement? KABOOM. The scheduler in the kernel can pick any thread it wants to execute.

Also,

  1. Shared variables are cause of all sorts of hard to track crashes in multi threaded code. Avoid it.
  2. Use message passing. Much cleaner way to doing everything shared variables can. Much easier to reason about.

Edit: About the question of enforcing ordering other than C++ synchronization built ins, you can use the synchronization primitives of the OS you are using windows, posix, whatever. That will be c functions and structs you will use but it technically works for C++ as well

Hemil
  • 916
  • 9
  • 27
  • 1
    Undefined behaviour doesn't mean that it can't *currently* do what you expect – Caleth Aug 12 '21 at 09:46
  • I've deliberately taken this example as I wanted to be sure about it being a data race or not when I'm making sure that client is sending requests only after getting the acknowledgement. Till now, from what I've read, I'm inclined to believe that it does fall in an undefined behaviour category. However I can't pinpoint exactly which requirement this program is failing to satisfy in order to be a well defined one. – Vishal Sharma Aug 12 '21 at 09:53
  • I dont think its undefined behavior right now @VishalSharma. I said you are walking on thin ice. If you need synchronization between requests in backend, you are already doing a lot of things wrong – Hemil Aug 12 '21 at 10:17
  • @Hemil this is just an example code written solely for the purpose of understanding little more about data races/undefined behaviour. I thought of this example only because I wanted to synchronise as much as possible without taking help of any OS/language's synchronisation primitives and see whether still there could be a data race. Clearly you don't agree with the other answer here. Atm, I'm not too sure as well. – Vishal Sharma Aug 12 '21 at 10:29
  • After reading happens-before and Inter-thread happens-before sections in https://en.cppreference.com/w/cpp/atomic/memory_order, it seems to me that it indeed is a data race. Because none of the points mentioned in those sections are equivalent to what was being done in the server program mentioned in the question. Like @Caleth mentioned in the comments, 'happens-before' is not about relative wall-clock time of events. – Vishal Sharma Aug 12 '21 at 11:02