0

I have a memory leak in my program and cannot understand why.

My program is multi-threaded and uses json-c and glib.

Thread 1 calls the following method every time it receives data over a UDP stream.

void foo(state_t *self, const char *data)
{
    char *x = strdup(data);
    g_async_queue_push(self->write_queue, x);
}

where write_queue is a GAsyncQueue initialized with g_async_queue_new().

Thread 2 processes the data in an endless loop.

static void bar (state_t *self)
{
    while (1)
    {
        void *msg = g_async_queue_pop(self->write_queue);
        parse_json ((char*)msg);
        free(msg);
    }
}

where

void parse_json (const char *data)
{
    struct json_object *elements;
    elements = json_tokener_parse(data);
    json_object_put(elements);
}

Threads are created with the g_thread_new() command.

When I run this program, I see memory consumption grow constantly and regularly with htop (roughly 100MB/sec).

When I comment out the call to parse_json(), the memory leak disappears.

When I run parse_json() in a unit test with a large for loop, I notice no memory leak.

What I am missing here?

NB: I notice that I get roughly the same volume of memory leak if I comment out both the call to parse_json() and the free(msg). This leads me to think that my code is keeping a reference count somewhere on the msg string.

I am using this version of Linux

Linux exp-infra 5.11.0-49-generic #55-Ubuntu SMP Wed Jan 12 17:36:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

I am compiling my code with g++ --std=c++20 -O2.

My version of g++ is 10.3.0.

Thanks!

olivier
  • 1
  • 1
  • When it comes to avoiding memory leaks, do yourself a favour and use a C++ library instead of a C library. Memory management will be a lot simpler. If you want to keep using [tag:json-c], please provide a [mre] so we can reproduce the memory leak. – Ted Lyngmo Mar 15 '22 at 21:42
  • Any chance of running this through this with valgrind or similar to help locate the leak? – user4581301 Mar 15 '22 at 22:07
  • It can happen that thread is unable to parse and process messages fast enough, so strings are getting queued. That can cause high memory consumption. Can you poll the queue size before you push and do not push if the queue is larger than N? – Dmytro Ovdiienko Mar 15 '22 at 23:10
  • `This leads me to think that my code is keeping a reference count somewhere on the msg string` That's not a concept in C. Either the memory is manually freed or not (there is not in-between where it has been freed but a reference is keeping it alive). – Martin York Mar 15 '22 at 23:14
  • 1
    Catch exceptions and see if that tells you something. Your code assumes you're getting complete JSON from your socket in a single read, which is highly unlikely. What does that parse method do when it receives incomplete JSON. The JSON library I use throws an exception, and then your dequeue code won't free the allocated memory. – Joseph Larson Mar 16 '22 at 00:14
  • Thank you all for your answers. @DmytroOvdiienko was correct, I capped the queue size and do not see the memory leak anymore. – olivier Mar 16 '22 at 07:49
  • @JosephLarson: that's a good point in general, but in my case, I know for sure that the json messages arrive in full because I have a check on that higher up in the code. Thanks again! – olivier Mar 16 '22 at 07:51

0 Answers0