Safely passing read-only data to a new thread

Question

Suppose I have a program that initializes a global variable for use by threads, like the following:

int ThreadParameter;

// this function runs from the main thread
void SomeFunction() {
    ThreadParameter = 5;

    StartThread(); // some function to start a thread
    // at this point, ThreadParameter is NEVER modified.
}

// this function is run in a background worker thread created by StartThread();
void WorkerThread() {
    PrintValue(ThreadParameter); // we expect this to print "5"
}

These questions should apply for any generic processor architecture that one might encounter. I want the solution to be portable - not specific to an architecture with stronger memory guarantees, like x86.

General question: despite being very common, is this really safe across all processor architectures? How to make it safe, if not?
The global variable isn't volatile; is it possibly going to be reordered after the StartThread() call and leave me hosed? How to fix this problem?
Suppose the computer has two processors that have their own caches. Main thread runs on first processor and worker thread runs on second processor. Suppose the memory block that contains ThreadParameter has been paged into each processor's cache before the program begins to run SomeFunction(). SomeFunction() writes 5 to ThreadParameter, which gets stored in the first processor's cache, and then starts the worker thread, which runs on the second processor. Won't WorkerThread() on the second processor see uninitialized data for ThreadParameter instead of the expected value of 5, since the memory page in the second processor hasn't yet seen the update from the first processor?
If something different is required - how best to handle this given that rather than a simple int, I could be working with a pointer to much more complex data types that aren't necessarily used in a multithreaded environment?

If my concerns are unfounded, what are the specific reasons why I don't need to worry?

SomeFunction is going to call multiple WorkerThreads, I assume. I think your question is more like: "Will simultaneous reads cause me an issue?" .... ? — Alex, Mar 21 '12 at 20:51
Well, when using POSIX threads you work around the global variable initialization conundrum using [`pthread_once()`](http://linux.die.net/man/3/pthread_once) -- I'm not sure how this translates to your definition of "portable". — Justin ᚅᚔᚈᚄᚒᚔ, Mar 21 '12 at 20:55
I imagine the best solution would be `boost:shared_mutex` and copying the read-only into each thread. — Pubby, Mar 21 '12 at 21:02
What about using a lock when you need to read from ThreadParameter? — YankeeWhiskey, Mar 21 '12 at 21:02
I apologise, but I need to understand this in steps. I have always believed that `volatile` is there so that a programmer can tell the C (It is C, right?) compiler that it should assume something else will touch an address. Is there a reason you don't want to use `volatile`? — gbulmer, Mar 21 '12 at 21:21
@gbulmer, you are right. volatile tells the compiler not to assume that the value of a variable will not change.. This eliminates some optimizations the compiler can do that cause problems during multithreading. Some compiler optimizations are eliminated, so there is a slight downside to using volatile. Since he is only reading this variable, he probably doesn't even have to make it volatile. The REAL answer is much more complicated though: http://stackoverflow.com/questions/2729453/c-thread-safe-integer has good info. — Alex, Mar 21 '12 at 21:32
@windfinder - I apologise for breaking protocol. I should have addressed James Johnston explicitly. I wanted to understand why the OP was avoiding using `volatile`. There may be a sophisticated reason, which I'd like to understand. Thanks for the link. I looked at it, and it is not clear to me yet that is the REAL answer to the OPs questions; I need more information to make that judgement. — gbulmer, Mar 21 '12 at 21:52
@gbulmer: it might not be easily possible. For example, suppose instead of a simple `int`, we are talking about a more complicated data structure, not originally designed for multithreading and not practical to modify, that has allocated some non-volatile memory during setup. Also, `volatile` incurs an unnecessary performance costs: the worker thread must always read from memory, when caching it in a register would suffice. — James Johnston, Mar 21 '12 at 22:00
@James Johnston "suppose instead of a simple int, ... a more complicated data structure, not originally designed for multithreading and not practical to modify" okay, that I buy. You'd like a portable way to force the write before the call. "`volatile` incurs an unnecessary performance costs" I agree, it is certainly not free, but it _may_ be cheaper than any portable synchronisation primitive, if that primitive has to be wrapped around every read in workers. Summary: it looks very desirable to take the cost before StartThread because changing the 'workers' may be much worse. Thank you. — gbulmer, Mar 21 '12 at 22:15

score 3 · Answer 1 · answered Mar 21 '12 at 21:58

When you create a new thread, the construction of the thread synchronizes with the start of the thread function. That means you're good - you write to ThreadParameter before creating the thread, and the threads access it after they start, so you can be sure that the write happens before the read, and so the threads are guaranteed to see the correct value.

(The compiler is required to ensure that all writes done before the thread is started are visible within the new thread.)

score 3 · Accepted Answer · answered Mar 22 '12 at 01:27

From you description, it seems you're writing to ThreadParameter (or some other data structure) BEFORE starting any child threads, and you will never write to ThreadParameter again... it exists to be read as needed, but never changed again after its initialization; is that correct? If so, then there's no need whatsoever to employ any thread synchronization system calls (or processor/compiler primitives) every time a child thread wants to read the data, or even the first time for that matter.

The treatment of volatile is somewhat compiler-specific; I know that at least with Diab for PowerPC, there is a compiler option regarding the treatment of volatile: either use the PowerPC EIEIO (or MBAR) instruction after every read/write to a variable, or don't use it... this is in addition to prohibiting compiler optimizations associated with the variable. (EIEIO/MBAR is PowerPC's instruction for prohibiting reordering of I/O by the processor itself; i.e, all I/O from before the instruction must complete before any I/O after the instruction).

From a correctness/safety standpoint, it doesn't hurt to declare it as volatile. But from a pragmatic standpoint, if you initialize ThreadParameter far enough ahead of StartThread(), declaring it volatile shouldn't really be necessary (and not doing so would speed up all subsequent accesses of it). Pretty much any substantial function call (say, perhaps to printf() or cout, or any system call, etc) would issue orders of magnitude more instructions than necessary to ensure there's no way the processor wouldn't have long ago handled the write to ThreadParameter before your call to StartThread(). Realistically, StartThread() itself almost certainly will execute enough instructions before the thread in question actually starts. So I'm suggesting that you don't really need to declare it volatile, probably not even if you initialize it immediately before calling StartThread().

Now as to your question regarding what would happen if the page containing that variable were already loaded into the cache of both processors before the processor running the main thread performs the initialization: If you're using a commonly available general purpose platform with like-kind CPUs, the hardware should already be in place to handle the cache coherency for you. The place you get into trouble with cache coherency on general purpose platforms, whether or not they're multiprocessor, is when your processor has separate instruction & data caches and you write self-modifying code: The instructions written to memory are indistinguishable from data, so the CPU doesn't invalidate those locations in the instruction cache, so there may be stale instructions in the instruction cache unless you subsequently invalidate those locations in the instruction cache (either issuing your own processor-specific assembly instructions, which you might not be allowed to do depending on your OS and your thread's privilege level, or else issuing the appropriate cache-invalidate system call for your OS). But what you're describing isn't self-modifying code, so you should be safe in that regard.

Your question 1 asks how to make this safe across ALL processor architectures. Well, as I discussed above, you should be safe if you're using like-kind processors whose data busses are properly bridged. General-purpose processors designed for multiprocessor interconnection have bus snoop protocols to detect writes to shared memory... as long as your threading library properly configures the shared memory region. If you're working in an embedded system, you may have to configure that yourself in your BSP... for PowerPC, you need to look at the WIMG bits in your MMU/BAT configuration; I'm unfamiliar with other architectures to give you pointers on those. BUT.... If your system is homebrew or if your processors are not like-kind, you may not be able to count on the two processors being able to snoop each others' writes; check with your hardware folks for advice.

If you are only writing ThreadParameter _once_ and, assuming you are using a well behaved language and compiler (C++), create the variable as late as possible, preferably as you initialize it, and make it **const**. If you'll be modifying ThreadParameter in the main thread & expecting the child threads to honor that change, try putting a wait condition on the variable. You'll wait for the main thread to write ThreadParameter before using it & can check for new, pending updates as the child thread runs. A well written procedure can back out of itself and rerun using the updated value. — Wes Miller, Mar 22 '12 at 18:17

score 1 · Answer 3 · answered Mar 22 '12 at 01:41

Yes, it is safe.
Don't know. Maybe : if( ThreadParameter = 5 ) StartThread();. However, in general, try not to second guess the compiler.
Probably not. If you had to worry about such low levels details when writing code, then the logic that controls how a program gets executed on a multi-core machine is probably not doing its job very well.
Boost is your friend for working with complex types in a multi-threaded environment.

Safely passing read-only data to a new thread

3 Answers3