Why is this simple libpqxx code leaking memory?

Question

Recently I wrote an application that accesses a Postgres DB via libpqxx, and it leaks memory badly. Even this simple test program, which is based on the example at http://pqxx.org/devprojects/libpqxx/doc/4.0/html/Reference/a00001.html leaks like no tomorrow.

(Edit: I added calls to commit() and clear() in response to suggestions. Same leakiness.)

#include <iostream>
#include <pqxx/pqxx>
#include <string>
#include <stdio.h>

int main()
{
    try
    {
        pqxx::connection c("user=postgres");

        int i = 0;          
        while(true)
        {
            pqxx::work w(c);
            pqxx::result r = w.exec("SELECT 1");
            w.commit();

            i++;
            if ( i % 1000 == 0 )
                printf( "Cycle %d\n", i );

            r.clear();
        } //while
    } //try
    catch (const std::exception &e)
    {
        std::cerr << e.what() << std::endl;
        return 1;
    } //catch
} //main

After about 75,000 cycles of the loop, top shows 206Mb of virtual memory usage, and it keeps climbing. I ran a similar test program with 5000 cycles through valgrind and got this:

==1647== 13,732,155 (219,868 direct, 13,512,287 indirect) bytes in 4,997 blocks are definitely lost in loss record 12 of 12
==1647==    at 0x40060D5: operator new(unsigned int) (vg_replace_malloc.c:214)
==1647==    by 0x404C0A9: pqxx::result::result(pg_result*, int, std::string const&, int) (in /usr/lib/libpqxx-4.0.so)
==1647==    by 0x40309EF: pqxx::connection_base::make_result(pg_result*, std::string const&) (in /usr/lib/libpqxx-4.0.so)
==1647==    by 0x4036D65: ??? (in /usr/lib/libpqxx-4.0.so)
==1647==    by 0x405EFD6: pqxx::transaction_base::DirectExec(char const*, int) (in /usr/lib/libpqxx-4.0.so)
==1647==    by 0x40416EA: pqxx::dbtransaction::do_exec(char const*) (in /usr/lib/libpqxx-4.0.so)
==1647==    by 0x40618FA: pqxx::transaction_base::exec(std::string const&, std::string const&) (in /usr/lib/libpqxx-4.0.so)
==1647==    by 0x80498F8: main (dbtest.cpp:21)

Any idea what's going on? It's very difficult to accept that a widely used library like libpqxx would have such a severe bug, so what might I be doing wrong here?

Configuration details:

• OS: Linux 2.6.18-238.el5
• gcc version 4.4.0
• libpqxx 4.0
• postgres 9.2

(Final edit: I found it easier to replace libpqxx with libpq than to keep investigating this memory leak.)

Answer 1

I could not reproduce your problem.

Neither with libpqxx-2.6.9 nor with libpqxx-4.0.1 (at this time the current stable version).

• Debian Linux 2.6.32
• gcc version 4.4.5
• libpq5 8.4.17
• libpqxx 4.0.1
• libstdc++ 6.4.4
• postgres 8.4

Memory usage is constant.

$ valgrind --tool=memcheck run.sh
[...]
Cycle 35000
Cycle 36000
^C==18420==
==18420== HEAP SUMMARY:
==18420== in use at exit: 0 bytes in 0 blocks
==18420== total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==18420==
==18420== All heap blocks were freed -- no leaks are possible

I would check, if you have a problem with different installations of the libraries you are using. For example, do you link against libpqxx-4.0, but later on, when you run the program, you are using a different version.

$ ldd client
linux-gate.so.1 => (0xb7776000)
libpqxx-4.0.so => not found
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7669000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb764a000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7503000) libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb74ea000)
libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb74c4000)
/lib/ld-linux.so.2 (0xb7777000)

As I have downloaded and installed libpqxx-4.0.1 manually, I have to set LD_LIBRARY_PATH explicitly, since I have not installed it below /usr/local/. If different versions are installed in /usr/local, you should cleanup there before.

In your case (using ldd) libpqxx should point to the version you have compiled and installed. Make sure that there is only one libpqxx installed on your system.

I used this Makefile

CPPFLAGS += -I/home/dev/data/src/cpp/libpqxx-4.0.1/install/include
LDFLAGS += -L/home/dev/data/src/cpp/libpqxx-4.0.1/install/lib -lpqxx -lstdc++

client: client.o

and libpqxx has been built with

./configure --prefix=/home/dev/data/src/cpp/libpqxx-4.0.1/install --enable-shared
make
make install

and I have run the program using

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/dev/data/src/cpp/libpqxx-4.0.1/install/lib client

Answer 2

It's likely that pqxx::result is leaking memory. Try adding:

r.clear();

It will at least reduce the leaking.

#include <iostream>
#include <pqxx/pqxx>
#include <string>
#include <stdio.h>

int main()
{
    try
    {
        pqxx::connection c("user=postgres");

        int i = 0;  

        while(true)
        {
            pqxx::work w(c);
            pqxx::result r = w.exec("SELECT 1");

            i++;
            if ( i % 1000 == 0 )
                printf( "Cycle %d\n", i );

            r.clear();
        } //while
    } //try
    catch (const std::exception &e)
    {
        std::cerr << e.what() << std::endl;
        return 1;
    } //catch
} //main

Answer 3

Try to declare pqxx::result r whitout the while. And call it inside, as you are doing. Because for each cycle of the while, you are creating a pqxx::result object. In this way you will create just one object to all the loop, to charge and clear in each cycle:

#include <iostream>
#include <pqxx/pqxx>
#include <string>
#include <stdio.h>

int main()
{
    try
    {
        pqxx::connection c("user=postgres");

        int i = 0;          
        pqxx::result r;
        while(true)
        {
            pqxx::work w(c);
            r = w.exec("SELECT 1");
            w.commit();

            i++;
            if ( i % 1000 == 0 )
                printf( "Cycle %d\n", i );

            r.clear();
        } //while
    } //try
    catch (const std::exception &e)
    {
        std::cerr << e.what() << std::endl;
        return 1;
    } //catch
} //main

Or even better, create a method to call in each cycle of the while, so, when each call of the method finish, tne temporary object will gone.

main()
{
    ...
    while()
    {
        ...
        method_to_call_pqxx();
    }
}