Thread Safety of datatstructures, Where to add synchronization primitives?

Question

This is a design question that has been bugging me for a while now. It is pretty simple really, when you provide datastructure libraries do you build in the thread-safety primitives or just provide the constructs and let the system using it decide on how to implement the actual operations.

A quick example, a Circular buffer which supports a Producer, Consumer model. 2 Methods, Get and Write, each updates a global variable fill count. Now, do you just provide the mutexes to lock and let the code using the buffer grab the mutexes OR do you the locking internally and provide mutual exclusion out of the box.

STL seems to take the approach of doing it externally, but there are performance reasons of why you would want to provide finer grained locking.

Thoughts ?

If you build thread-safety directly into your data structures not only are you closely coupling two separate design concepts but you will likely kill some performance. — AJG85, Feb 03 '12 at 16:45
http://stackoverflow.com/questions/3687740/coding-style-lock-unlock-internal-or-external — Bojan Komazec, Feb 03 '12 at 16:46
@AJG85 ok the performance argument, if you built it into the datastructure you could actually design finer grained locking primitives. I would argue the performance might be better. — creatiwit, Feb 03 '12 at 17:31
@shrin: I simply meant acquiring a lock requires additional resources. In a single threaded application or in certain types of containers you may end up paying for what you don't need. — AJG85, Feb 03 '12 at 18:38

score 3 · Answer 1 · answered Feb 03 '12 at 16:48

IMHO there is no clear winner to this discussions. There are pros and cons on either side of the board:

Having synchronization as a part of the API (inside the module):

Ensures that the caller won't have to spend too much time thinking about synchronization
Ensures caller won't make a mistake synchronizing (as it can be quite error prone to synchronize in languages that don't have in-built support for synchronization constructs)
You can do finer grained locking and optimize your library

Letting the caller synchronize

Gives the caller more control
In a single-threaded program the caller does not spend the time locking/unlocking

You could make a decision based on the case:

If it is a library that would most probably be used in a multi-threaded situation then provide in-built locking
If locking is going to be very tedious to implement (like a per node locking for a threadsafe queue) then provide it as a part of the library
Consider providing 2 versions of your library - locked and unlocked. Use templates in C++ to provide that with a nice syntax coupled with concurrent patterns such as ThreadSafe Interface
Be consistent! If you are providing a set of modules in a library make sure you are consistent in your syntax for threadsafe vs. non-threadsafe modules. This is my peeve with Java Swing that they are not consistent. Certain parts of the library are threadsafe while others aren't.

Hope this helps!

score 2 · Answer 2 · answered Feb 03 '12 at 16:43

2

If you can, do no locking at all.

If there is no way around you have 2 options: (1) internal locking (2) external locking

(1) Best way to do it is internal locking. (2) Another way is to let the user tackle the concurrency issues.

Either way you go, you must document your class to let users/callers know how it handles concurrency.

Here is the summary from Effective Java:

To summarize, every class should clearly document its thread safety properties with a carefully worded prose description or a thread safety annotation. The synchronized modifier plays no part in this documentation. Conditionally thread-safe classes must document which method invocation sequences require external synchronization, and which lock to acquire when executing these sequences. If you write an unconditionally thread-safe class, consider using a private lock object in place of synchronized methods. This protects you against synchronization interference by clients and subclasses and gives you the flexibility to adopt a more sophisticated approach to concurrency control in a later release

answered Feb 03 '12 at 16:43

Adrian

5,603
8
53
85

"Best way to do it is internal locking." Why is it the best? Best for what? When? If I'm using a data structure in a single thread I don't want it to waste time playing around with locks. – R. Martinho Fernandes Feb 03 '12 at 16:45
That's why I said if you can, use no locking at all. Then try internal locking, then external. AND document your class – Adrian Feb 03 '12 at 16:47
I think no locking is a sub-class of internal locking. in both cases, the library takes care of things, and the caller need not worry. The ability to use lock-free algorithm is indeed a pro for internal locking, but I don't think it's a definite advantage. – ugoren Feb 03 '12 at 18:16
@ugoren I disagree because IT HAS TO BE DOCUMENTED WITHIN THE CLASS. If we didn't have documentation, then yea it would be "a subclass" like you say because you just don't know if something is thread safe or not, but the documentation tells you. Java does the same thing; for exmaple see ConcurrentMap and Map. – Adrian Feb 03 '12 at 18:27
I'm not sure we agree on terms. "No locking" is when the library is thread-safe, despite using no locks. Internal locking is when the library uses locks to be thread safe. External locking is when it's the callers responsibility to call the class under appropriate locks (or simply in a single thread). I say the first is a subclass of the second. Regarding documentation, the user doesn't need to care about the difference between them. – ugoren Feb 03 '12 at 21:40

score 2 · Accepted Answer · answered Feb 03 '12 at 17:17

2

I was thinking about the exact problem sometime ago. So I went ahead and wrote some sample code to understand the advantages and disadvantages of various approaches. So, instead of giving a theoretical answer, let me give you some code for the same problem that you have mentioned in the OP i.e. circular buffer (queue) with multiple producer and consumer.

Here it is.

Perhaps looking at the code may give you some clarification. I shall add more points if need be.. but for now, look at the code and derive the obvious!

answered Feb 03 '12 at 17:17

Sangeeth Saravanaraj

16,027
21
69
98

Your code is a good example, you grab a mutex for the entire methods and within While loops. Now that is the exact behavior that I want to avoid with internal locking. In a circular queue all you really need to lock is how the fill count is updated, thats abt it – creatiwit Feb 03 '12 at 18:06
@shrin Now you are talking my friend.. that is good! Writing example programs help me understand some design problems. I'm glad it helped you in some form! Thanks! :) – Sangeeth Saravanaraj Feb 03 '12 at 18:09
@sangeeth-saravanarj yep i went through the same process, with internal locking there is one issue. Say you have internal Enqueue wait until there is space available and Dequeue wait if empty. Now, only the external entity knows if there is more data that might enter the queue, so you end up in a spot where the queue always busy waits – creatiwit Feb 03 '12 at 18:13
Yes, I agree. There will be some race conditions in this scenario. However when I ran this program with 100 producers and 100 consumers for overnight with a queue size of just 100, I didn't face any dead-/live-lock. That is the beauty of the multi-threaded programming, its very hard to test the corner cases! – Sangeeth Saravanaraj Feb 03 '12 at 18:25

score 1 · Answer 4 · answered Feb 03 '12 at 16:48

Well there are two important issues to consider here:

Are the operations going to be used individually or can they be used together in certain scenarios?
Is it possible that these operations might also be used in a single-threaded environment?

Point 1 carries some interesting implications. If you lock internally, then if you only use each operation by itself, you are safe. However, if you might use two or more of them in a sequence, remember that atomicity of each operation does not guarantee atomicity of the entire sequence, so outside locking is needed in any case. For example:

if(buffer not empty)
    extract from buffer

Even though each of these two operations is atomic by itself, the above code is not thread safe for obvious reasons.

Point 2 is again an argument against internal locking: in a single-threaded environment you don't need the locks, therefore you are incurring unnecessary overheads by acquiring and releasing the internal locks. This is one of the reasons for which the HashTable and Vector classes have been deprecated in Java for example.

score 1 · Answer 5 · edited Jun 20 '20 at 09:12

Herb Sutter and Andrei Alexandrescu advise the following [source]:

If your application shares data across threads, do so safely:

Consult your target platforms' documentation for local synchronization primitives

Prefer to wrap the platform's primitives in your own abstractions

Ensure that the types you are using are safe to use in a multithreaded program

Guarantee that unshared objects are independent

Document what the caller needs to do to use the same object of that type in different threads

This article discusses three ways of thread-safety design: internal, external and lock-free so you might find it useful.

score 1 · Answer 6 · answered Feb 16 '12 at 11:44

If there is only one thread doing reading and only one thread doing writing no synchronization is necessary on the condition that the head and tail indeces are updated by writing their respective new values in one operation:

// adding single bytes

i=circ.head;
circ.buffer[i]=chr;
++i;
if (i==circ.limit) i=0;
circ.head=i;

// removing single bytes

i=circ.tail;
if (i!=circ.head)    /* there's data in the buffer */
{
  chr=circ.buffer[i];
  ++i;
  if (i==circ.limit) i=0;
  circ.tail=i;
}

By calculating the new index value outside of the circ structure you ensure that you won't confuse the other thread with partial values: if cirf.tail is incremented directly, tested for limit and perhaps also cleared the thread using circ.head runs the risk of having two different values for tail available to it for comparison.

If there are multplie threads reading and multiple threads writing I suggest you use a spinlock since the times necessary for the operations themselves are probably very short.

score 0 · Answer 7 · answered Feb 03 '12 at 18:53

If your methods to handle your data structure are small, just some instructions, you shouldn't do locking at all. Atomic operations are the way to go, here. Both, C++11 and C11, provide new interfaces for that. Many compilers already have such interfaces as extensions to the previous versions of the standards.

Thread Safety of datatstructures, Where to add synchronization primitives?

7 Answers7