2

In a multi-threaded C program I used GList functionality of GLib (https://developer.gnome.org/glib/2.35/glib-Doubly-Linked-Lists.html#g-list-append) where multiple threads created their own lists. I observed unpredictable crashes, sometimes as soon as the application loads. The stack trace shows some crash in glist_* functions some message like this:

(gdb) bt
#0  0x00007fffeb54a964 in g_slice_alloc () from /lib64/libglib-2.0.so.0
#1  0x00007fffeb52aac6 in g_list_append () from /lib64/libglib-2.0.so.0

Or messages like this:

MEMORY-ERROR: [25628]: GSlice: assertion failed: sys_page_size == 0 Aborted (core dumped)

(process:15426): GLib-ERROR (recursed) **: gmem.c:157: failed to allocate 137438953456 >bytes aborting... Aborted (core dumped)

I have reasons to believe that introduction of GList caused all this crashes. In a single threaded program, I have never seen these issues.

Is GList inherently thread-safe? If not, what I need to do?

Community
  • 1
  • 1
Soumya Kanti
  • 1,429
  • 1
  • 17
  • 28

4 Answers4

1

Are you using GThread?:

After calling g_thread_init(), GLib is completely thread safe.

Take a look to this page

David Ranieri
  • 39,972
  • 7
  • 52
  • 94
  • I do not, I used POSIX threads (pthread_create etc). Should I use g_thread_init(), with NULL as argument, from parent thread, before spawning child threads? – Soumya Kanti May 16 '13 at 09:52
  • 1
    No, you have to use the g_thread family functions, on the other hand `G_THREADS_IMPL_POSIX` is defined if POSIX style threads are used – David Ranieri May 16 '13 at 10:09
  • 1
    Keep reading: "GLib is completely thread safe (all global data is automatically locked), but individual data structure instances are not automatically locked for performance reasons. So, for example you must coordinate accesses to the same GHashTable from multiple threads. The two notable exceptions from this rule are GMainLoop and GAsyncQueue, which are threadsafe and need no further application-level locking to be accessed from multiple threads." You said threads *created* their own lists, but are they accessed concurrently from multiple threads? – nemequ May 17 '13 at 05:17
  • Hi nemequ, thanks for your comment. In my case, threads created their own lists and are accessed from the same thread only, no concurrent access from multiple thread takes place. – Soumya Kanti May 17 '13 at 08:50
  • Hi David RF, thanks for your input. I may sound naive, but do you mean to say that if I continue to use POSIX style threads, I need to define `G_THREADS_IMPL_POSIX` first? – Soumya Kanti May 17 '13 at 08:53
  • @SoumyaKanti No, `if G_THREADS_IMPL_POSIX == 1 then POSIX style threads are used`, anyway, seems that this macro is [deprecated](https://developer.gnome.org/glib/2.31/glib-Deprecated-Thread-APIs.html) – David Ranieri May 17 '13 at 09:00
  • 2
    Warning g_thread_init has been deprecated since version 2.32 and should not be used in newly-written code. This function is no longer necessary. The GLib threading system is automatically initialized at the start of your program. Sorry, I was reading an old documentation :( – David Ranieri May 17 '13 at 09:02
0

First of all the debugging symbols might help. The detailed description of how to get them is described on gnome live and depends on which distribution you are using.

As noted before the g_slice API is thread-safe. IIRC it was designed to be lock-free or very close to that. GLib data structures generally are safe to use in multithread environments as noted (as long as instances are not accessed from multiple threads at the same time) - if they are not then it is a bug which should be reported upstream (however as GLib is widely used including multithread environments it is very unlikely that it have some obvious bugs).

Given the stacktrace it looks like memory corruption. My guess it that you have a buffer overflow/underflow somewhere and write over g_slice internal memory or you use uninitialized GList pointer with similar effect of accessing 'random memory' OR you try to pass a negative value to allocate for some reasons which results with overflow of integer. I would suggest to run with G_SLICE=always-malloc under Valgrind. If it is too slow there are other methods like AddressSanitizer in gcc 4.8+ and clang (I don't remember from which version). Please note that such bugs might not be in code related to GList but caused by subtle interaction (different address layout etc.).

Once you have done it (use debug symbols and run with valgrind) you should have much better idea on what is going on and where the bug might be. Please note that it wouldn't get all bugs but it would help with most common cases.

Maciej Piechotka
  • 7,028
  • 6
  • 39
  • 61
  • Simple data structures such as GList are **not** thread-safe by default. If you look into the source code you'll see that there is no concurrency protection. – Ancurio May 18 '13 at 19:17
  • @Ancurio: Sorry. Clarified - I meant the API not the instances (i.e. are safe to use in multithread environment not are thread-safe). – Maciej Piechotka May 18 '13 at 19:21
0

referring to doc from this page:

The general policy of GLib is that all functions are invisibly threadsafe with the exception of data structure manipulation functions, where, if you have two threads manipulating the same data structure, they must use a lock to synchronize their operation. GLib creates a worker thread for its own purposes so GLib applications will always have at least 2 threads.

Sniper Law
  • 77
  • 1
  • 2
-1

GList, as most other simple GLib data structures, is not thread safe. However, since you're not concurrently modifying the same list from multiple threads, there shouldn't be a problem with this. I think what is actually causing your segfaults are multiple concurrent calls to g_slice_alloc (resulting from the g_list_append/prepend calls). Other people seem to have bumped into the same problem before.

The way I would work around this issue is by creating a mutex protected wrapper function for each g_list one that allocates memory that you're using (let's take g_list_append as an example), and write:

/* Init this before starting your threads with g_mutex_new() */
static GMutex *g_list_mutex;

GList *safe_list_append(GList *list, gpointer data)
{
    GList *ret;

    g_mutex_lock(g_list_mutex);
    ret = g_list_append(list, data);
    g_mutex_unlock(g_list_mutex);

    return ret;
}

(I didn't test this)

Note that you need to use the same mutex across all wrapper functions, so only one thread ever gets into the slice function at once.

Community
  • 1
  • 1
Ancurio
  • 1,698
  • 1
  • 17
  • 32
  • g_slice api should be thread safe (see nemequ comment, IIRC it is lock-free internally) - if it crashes then a) there is bug in glib which should be reported upstream (unlikely given the widespread of usage using mulithread programs) on https://bugzilla.gnome.org/ or b) there is bug in program intself - as liked problem also occured with `G_SLICE=always-malloc` it is far more _likely_, otherwise it would mean that but is in both system malloc and slice. As of your solution - you protect only the single calls so the freeing the memory and appending will be done concurrently. – Maciej Piechotka May 18 '13 at 18:53
  • So even if there would be a problem in slice API and malloc it would NOT protect it against it. – Maciej Piechotka May 18 '13 at 18:54
  • >"you protect only the single calls so the freeing the memory and appending will be done concurrently" - I don't understand this, can you clarify? – Ancurio May 18 '13 at 19:00
  • Fixed by the edit. However my point about fixing wrong problem in wrong place still stand. Usually the system libraries (and GLib is one of them) is the last place you should look for bug as they are usually very well tested (both by unit tests and widespread deployment). Even if they *are* bugs they should be reported upstream and not workaround in your code as a) you make your code much slower then it is intended to be and b) fixing in upstream would solve the problem for everyone not just you. – Maciej Piechotka May 18 '13 at 19:10
  • Yeah, at first I was just writing the example with one function in mind, but then I updated it in case OP uses other g_list calls. Also, in case it is really an upstream issue, usually you still want to have a workaround to use in the meantime. – Ancurio May 18 '13 at 19:14