Profile shared object without recompiling main program

Question

I currently am developing a shared lilbrary for loading into PostgreSQL (as C-Language functions, see here). Now I would like to profile the function in this library without recompiling PostgreSQL itself.

I tried callgrind using

valgrind --tool=callgrind path/to/postgres arguments-to-postgres

This gives me profiling information for PostgreSQL itself, but fails to log the shared library I am interested in.

I also tried sprof, but I have no idea how to get that one working.

Any ideas would be highly appriciated.

P.S.: Please do not suggest just pausing the application in a debugger. With function runtimes way below 0.01 seconds I need more detailled results.

Just a short update: Upgrading to PostgreSQL 9.0 and Ubuntu 11.4 did the trick. Now everything works as expected. — Thilo, Aug 22 '11 at 11:47

score 2 · Answer 1 · answered Jul 27 '11 at 10:17

Using callgrind should work as expected. To test this I setup a simple project using a simple library and main functionMakefile:

CFLAGS=-fpic
exe:exe.o lib.so
        cc -o exe exe.o lib.so
lib.so:lib.o
        cc -shared lib.o -o lib.so
clean:
        rm -f exe lib.so *.o

lib.c is a simple library containing 2 functions:

#include <stdio.h>
void someOtherFunction() { printf("someOtherFunction\n"); }
void someFunction() { printf("someFunction\n"); someOtherFunction(); }

exe.c is a very simple executable:

int someFunction();
void main() { someFunction(); }

Use the Makefile to build the executable and run it using valgrind like so:

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PWD valgrind --tool=callgrind ./exe

If you examine the callgrind output, you will find the profiling data for both functions in the shared library. If you cannot see these functions you may be using a non-standard environment that does not support this functionality. I'm using Linux Mint 11 x64, with the latest patches.

Thanks for your input. This minimal example works. However, in my case above there are two major differences: The library is not linked at compile time, but at runtime using `dlopen`. Furthermore, my Postgres build has no debugging information enabled. Thus, callgrind seems to get kinda lost within the called functions. — Thilo, Jul 28 '11 at 05:05

thiton · Answer 2 · 2011-07-29T08:43:03.373

1

Point 1: Valgrind seems to have issues with privilege escalation, which is VERY likely to be performed by Postgres, see http://www.mail-archive.com/valgrind-users@lists.sourceforge.net/msg02355.html

Point 2: Have you tried to prove (e.g. with strace) that your SL is actually loaded in the same process? Have you tried to --trace-children=yes?

Point 3: I've tried to modify the test by compiling exe.o and exe with -g0 and using dlopen to load the file, i.e.:

all: exe lib.so
exe : exe.c 
        cc -g0 -o exe exe.c -ldl

lib.so:lib.c
        cc -shared lib.c -o lib.so
clean:
        rm -f exe lib.so *.o

and

#include 
#include 

void main() { 
    void *handle;
    void (*p)();
    int i;

    handle = dlopen("./lib.so", RTLD_LAZY);
    if ( ! handle ) { printf("Object not found\n"); return; }
    p = dlsym(handle, "someFunction");
    if ( ! p ) { printf("Function not found\n"); return; }
    for (i = 0; i < 100; ++i) (*p)();
    dlclose(handle);
}

Works with callgrind. Maybe Postgres is not using -ldl to open the object files?

edited Jul 29 '11 at 08:43

answered Jul 29 '11 at 08:27

thiton

35,651
4
70
100

Thanks for your ideas. Regarding Point 2: The strace output does contain only one call to execve, which as far as I understand suggests that everything is running in the same process. However, adding --trace-children=yes has no effect. I will have a look at your point 1 later, now my time is running short. – Thilo Jul 29 '11 at 10:20
Got time now to look into your first point. This also seems very likely for me to be the cause of trouble. Which leaves one question: Any other suggestions? – Thilo Aug 02 '11 at 03:20
Running postgres as root to prevent privilege escalation? – thiton Aug 02 '11 at 06:47
Postgres prohibits running as root. However, I can start it in single user mode with it's own user. This does not solve the problem, however. – Thilo Aug 02 '11 at 08:00
Does it still call setuid()? If yes, you might be able to solve the issue by LD_PRELOAD -ing away the setuid call. – thiton Aug 02 '11 at 12:37
Good question. Just had a short look: It does not call setuid(). – Thilo Aug 03 '11 at 05:14
Well, fair enough. I am out of ideas, sorry :-(. – thiton Aug 03 '11 at 12:24
Still, thank you for your ideas. There were quite a few nice ones! – Thilo Aug 03 '11 at 12:37

Profile shared object without recompiling main program

2 Answers2