Reader/Writer implementation in C

Question

I'm currently learning about concurrency at my University. In this context I have to implement the reader/writer problem in C, and I think I'm on the right track.

My thought on the problem is, that we need two locks rd_lock and wr_lock. When a writer thread wants to change our global variable, it tries to grab both locks, writes to the global and unlocks. When a reader wants to read the global, it checks if wr_lock is currently locked, and then reads the value, however one of the reader threads should grab the rd_lock, but the other readers should not care if rd_lock is locked.

I am not allowed to use the implementation already in the pthread library.

typedef struct counter_st {
    int value;
} counter_t;

counter_t * counter;
pthread_t * threads;

int readers_tnum;
int writers_tnum;

pthread_mutex_t rd_lock;
pthread_mutex_t wr_lock;

void * reader_thread() {

    while(true) {
        pthread_mutex_lock(&rd_lock);
        pthread_mutex_trylock(&wr_lock);
        int value = counter->value;
        printf("%d\n", value);
        pthread_mutex_unlock(&rd_lock);
    }
}

void * writer_thread() {

    while(true) {
        pthread_mutex_lock(&wr_lock);
        pthread_mutex_lock(&rd_lock);

        // TODO: increment value of counter->value here.
        counter->value += 1;

        pthread_mutex_unlock(&rd_lock);
        pthread_mutex_unlock(&wr_lock);
    }
}

int main(int argc, char **args) {

    readers_tnum = atoi(args[1]);
    writers_tnum = atoi(args[2]);

    pthread_mutex_init(&rd_lock, 0);
    pthread_mutex_init(&wr_lock, 0);

    // Initialize our global variable
    counter = malloc(sizeof(counter_t));
    counter->value = 0;

    pthread_t * threads = malloc((readers_tnum + writers_tnum) * sizeof(pthread_t));
    int started_threads = 0;

    // Spawn reader threads
    for(int i = 0; i < readers_tnum; i++) {
        int code = pthread_create(&threads[started_threads], NULL, reader_thread, NULL);

        if (code != 0) {
            printf("Could not spawn a thread.");
            exit(-1);
        } else {
            started_threads++;
        }
    }

    // Spawn writer threads
    for(int i = 0; i < writers_tnum; i++) {
        int code = pthread_create(&threads[started_threads], NULL, writer_thread, NULL);

        if (code != 0) {
            printf("Could not spawn a thread.");
            exit(-1);
        } else {
            started_threads++;
        }
    }
}

Currently it just prints a lot of zeroes, when run with 1 reader and 1 writer, which means, that it never actually executes the code in the writer thread. I know that this is not going to work as intended with multiple readers, however I don't understand what is wrong, when running it with one of each.

Answer 1

Don't think of the locks as "reader lock" and "writer lock".

Because you need to allow multiple concurrent readers, readers cannot hold a mutex. (If they do, they are serialized; only one can hold a mutex at the same time.) They can take one for a short duration (before they begin the access, and after they end the access), to update state, but that's it.

Split the timeline for having a rwlock into three parts: "grab rwlock", "do work", "release rwlock".

For example, you could use one mutex, one condition variable, and a counter. The counter holds the number of active readers. The condition variable is signaled on by the last reader, and by writers just before they release the mutex, to wake up a waiting writer. The mutex protects both, and is held by writers for the whole duration of their write operation.

So, in pseudocode, you might have

Function rwlock_rdlock:
    Take mutex
    Increment counter
    Release mutex
End Function

Function rwlock_rdunlock:
    Take mutex
    Decrement counter
    If counter == 0, Then:
        Signal_on cond
    End If
    Release mutex
End Function

Function rwlock_wrlock:
    Take mutex
    While counter > 0:
        Wait_on cond
End Function

Function rwlock_unlock:
     Signal_on cond
     Release mutex
End Function

Remember that whenever you wait on a condition variable, the mutex is atomically released for the duration of the wait, and automatically grabbed when the thread wakes up. So, for waiting on a condition variable, a thread will have the mutex both before and after the wait, but not during the wait itself.

Now, the above approach is not the only one you might implement.

In particular, you might note that in the above scheme, there is a different "unlock" operation you must use, depending on whether you took a read or a write lock on the rwlock. In POSIX pthread_rwlock_t implementation, there is just one pthread_rwlock_unlock().

Whatever scheme you design, it is important to examine it whether it works right in all situations: a lone read-locker, a lone-write-locker, several read-lockers, several-write-lockers, a lone write-locker and one read-locker, a lone write-locker and several read-lockers, several write-lockers and a lone read-locker, and several read- and write-lockers.

For example, let's consider the case when there are several active readers, and a writer wants to write-lock the rwlock.

The writer grabs the mutex. It then notices that the counter is nonzero, so it starts waiting on the condition variable. When the last reader -- note how the order of the readers exiting does not matter, since a simple counter is used! -- unlocks its readlock on the rwlock, it signals on the condition variable, which wakes up the writer. The writer then grabs the mutex, sees the counter is zero, and proceeds to do its work. During that time, the mutex is held by the writer, so all new readers will block, until the writer releases the mutex. Because the writer will also signal on the condition variable when it releases the mutex, it is a race between other waiting writers and waiting readers, who gets to go next.