Why does Valgrind show increasing stack usage with boost::thread?

Question

Wrote a simple test:

#include <iostream>
#include <boost/thread.hpp>

using namespace std;

void myThreadRun() {
    cout << "Thread id: " << boost::this_thread::get_id() << "\n";
}

int main() {
    for (int i = 0; i < 10000; i++) {
        boost::thread t(myThreadRun);

        t.join();
    }

    return 0;
}

on which Valgrind Massif shows the following graph:

Valgrind Massif profiling result for the above example

(The stack profiling was enabled. Platform: Linux Ubuntu x86).

This program does not actually seem to have memory leaks: the memory usage is stable.

I wonder: is it a problem of Valgrind or boost::thread? Or maybe I misinterpret something?

How would you explain that?

Do you see the same thing without any `std::cout` in the thread code? — James, Sep 14 '12 at 15:12
Oh, also, might be this: http://stackoverflow.com/questions/6321602/boost-thread-leakage-c — James, Sep 14 '12 at 15:14
Valgrind might replace the thread handling, so this particular data may not actually represent the real program profile. — Kerrek SB, Sep 14 '12 at 15:14
I have reproduced this and tried several things to try to fix or at least alter this behavior, with no luck yet. I'm still wondering if valgrind is doing something weird behind the scenes. — engineerC, Oct 05 '12 at 14:44
See if the leak is a multiple of your thread stack size. I remember this being an issue in some server software I was tasked to fix back in the day... The solution was to keep threads around blocked on some form of IPC and send them work via that same IPC method. — JimR, Oct 06 '12 at 05:16
What version of boost, pthreads, and valgrind are you using? — engineerC, Oct 08 '12 at 21:45

engineerC · Answer 1 · 2012-10-07T21:02:34.993

This isn't boost::threads, it happens with just plain pthreads too. I grabbed the sample program from here (Pthread Creation and Termination), upped the thread count to 1000 and compiled as plain C, and I see the same behavior when processing it with massif. So either it's something in pthreads or something valgrind/massif is doing.

EDIT: used program (Pthread Joining) as well. See second graph.

Creation and Termination:

    KB
547.6^                                                                       #
     |                                                                    @@@#
     |                                                                @@@@@@@#
     |                                                             @@@@@@@@@@#
     |                                                          @@@@@@@@@@@@@#
     |                                                      ::::@@@@@@@@@@@@@#
     |                                                  ::::: ::@@@@@@@@@@@@@#
     |                                               @@@::::: ::@@@@@@@@@@@@@#
     |                                           @@@@@@@::::: ::@@@@@@@@@@@@@#
     |                                        @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |                                     @@@@@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |                                @@@@@@@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |                            @@@@@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |                         @@@@@ @@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |                     ::@@@@@@@ @@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |                  @@@::@ @@@@@ @@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |               @@@@ @::@ @@@@@ @@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |           @@@@@@ @ @::@ @@@@@ @@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |      :::::@@@ @@ @ @::@ @@@@@ @@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
     |   :@@: :: @@@ @@ @ @::@ @@@@@ @@ @@ @@ @@@@@@@@@@::::: ::@@@@@@@@@@@@@#
   0 +----------------------------------------------------------------------->Mi
     0                                                                   13.22

Pthread joining, minus the math busy work:

    KB
548.8^                                                             #
     |                                                           @@#::
     |                                                        :::@@#::
     |                                                     ::::::@@#:::
     |                                                  ::::: :::@@#:::::
     |                                              @@@@::::: :::@@#:::::
     |                                            @@@@@ ::::: :::@@#:::::::
     |                                        :@@:@@@@@ ::::: :::@@#:::::::@
     |                                     @@@:@ :@@@@@ ::::: :::@@#:::::::@
     |                                  :::@ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |                               @@@:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |                            @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |                        @:::@@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |                     ::@@:: @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |                  ::@: @@:: @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |               :@@::@: @@:: @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |            @@@:@ ::@: @@:: @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |         @@@@@ :@ ::@: @@:: @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |     @@@:@@@@@ :@ ::@: @@:: @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
     |   @@@@ :@@@@@ :@ ::@: @@:: @@:@ @:: @ @:@ :@@@@@ ::::: :::@@#:::::::@::
   0 +----------------------------------------------------------------------->Mi
     0                                                                   19.14

It looks like joining should eventually bring the stack size back down, even under valgrind.

That example program neither detaches nor joins its threads, so it *should* leak. His program joins its threads, so it shouldn't. There is no sense comparing the two. — David Schwartz, Oct 06 '12 at 22:23
Don't you find it interesting, then, that the behavior is so similar? — engineerC, Oct 07 '12 at 20:57

score -1 · Answer 2 · answered Sep 24 '12 at 17:50

-1

Your code doesn't give the cleanups a chance to happen. When you call join on a thread, it waits until the thread signals completion, not the actual release of all of its resources. If you create the threads more slowly or put a delay or yield in your loop, the "leak" will go away.

answered Sep 24 '12 at 17:50

David Schwartz

179,497
17
214
278

Added `boost::this_thread::sleep(boost::posix_time::milliseconds(10))` into the loop and `boost::this_thread::sleep(boost::posix_time::seconds(20))` after the loop (before program exit). The chart stays completely the same. – weekens Sep 25 '12 at 07:23

Why does Valgrind show increasing stack usage with boost::thread?

2 Answers2