4

I am debugging the CPython executable by GDB and can't get the value of some variables despite of disabling all GCC optimizations:

(gdb) print *co
value has been optimized out

(gdb) frame
#0  _PyEval_EvalFrameDefault (
    f=Frame 0x7ffff7f36050, for file <frozen importlib._bootstrap>, line 8, in <module> (), 
    throwflag=<optimized out>) at Python/ceval.c:1057
1057            switch (opcode) {

(gdb) info locals
stack_pointer = 0x7ffff7f361c8
next_instr = 0x555555aff422
opcode = 100
oparg = 0
fastlocals = 0x7ffff7f361c8
freevars = <optimized out>
retval = 0x0
tstate = <optimized out>
co = <optimized out>
instr_ub = -1
instr_lb = 0
instr_prev = -1
first_instr = <optimized out>
names = <optimized out>
consts = <optimized out>
... <output cropped>

OS: Ubuntu 18.04.2 LTS
CPython version: 3.8.0a
Compiled by: GCC 7.3.0 on Linux
Debugger: GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git

Firstly, I did the usual compilation with --with-pydebug option.

./configure --with-pydebug
make -s -j

It should preserve all values for debugger but it doesn't.

I have read, that the cause of this problem is the compiler optimization, so I decided to disable it completely. For this I changed relevant lines in the configure script from:

# Optimization messes up debuggers, so turn it off for
# debug builds.
        if "$CC" -v --help 2>/dev/null |grep -- -Og > /dev/null; then
            OPT="-g -Og -Wall"
        else
            OPT="-g -O0 -Wall"
        fi

to:

# Optimization messes up debuggers, so turn it off for
# debug builds.
#                if "$CC" -v --help 2>/dev/null |grep -- -Og > /dev/null; then
#                    OPT="-g -Og -Wall"
#                else
#                    OPT="-g -O0 -Wall"
#                fi
                OPT="-g -O0 -Wall"

The resulting optimization options in the Makefile are:

$ grep -- '-O' Makefile
OPT=        -g -O0 -Wall
    $(MAKE) all CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov"
        $(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
        $(PYTHON_FOR_BUILD) -Wi -OO $(DESTDIR)$(LIBDEST)/compileall.py \
        $(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
        $(PYTHON_FOR_BUILD) -Wi -OO $(DESTDIR)$(LIBDEST)/compileall.py \

Also, didn't help.

The question: Why "the value has been optimized out" persists even with the compiler optimization disabled and how can I overcome this problem?

EDIT

(gdb) info sharedlibrary 
From                To                  Syms Read   Shared Object Library
0x00007ffff7dd5f10  0x00007ffff7df4b20  Yes         /lib64/ld-linux-x86-64.so.2
0x00007ffff7bbbbb0  0x00007ffff7bca0f1  Yes         /lib/x86_64-linux-gnu/libpthread.so.0
0x00007ffff79b2e50  0x00007ffff79b3bde  Yes         /lib/x86_64-linux-gnu/libdl.so.2
0x00007ffff77afe70  0x00007ffff77b093a  Yes         /lib/x86_64-linux-gnu/libutil.so.1
0x00007ffff741ca80  0x00007ffff74db2f5  Yes         /lib/x86_64-linux-gnu/libm.so.6
0x00007ffff70412d0  0x00007ffff71b9c3c  Yes         /lib/x86_64-linux-gnu/libc.so.6
MiniMax
  • 983
  • 2
  • 8
  • 24
  • Are you possibly debugging system-installed Python binary instead of the one you actually built? – Employed Russian Jun 30 '19 at 16:09
  • @EmployedRussian Nope. I run `gdb --args ./python my_code.py` from the directory, where the `python` was built. – MiniMax Jun 30 '19 at 16:30
  • @EmployedRussian May be it is a bug somewhere. I found this: [Bug 556975 - Variables optimized out (python)](https://bugzilla.redhat.com/show_bug.cgi?id=556975). Despite I am not on Fedora, the problem is the same. – MiniMax Jun 30 '19 at 16:33
  • I wonder if your `python` binary links `libpython.so` from system location (instead of a local copy). GDB `info shared` will answer that. – Employed Russian Jun 30 '19 at 16:42
  • @EmployedRussian Added the `info sharedlibrary` output into the question. – MiniMax Jun 30 '19 at 16:51
  • 1
    Could you please ensure that your CFLAGS are actually passed correctly to the compiler for building every `.o` file? You can use the [flagsup](https://github.com/ispras/flagsup) tool I wrote for that. Alternatively, you can try saving and reviewing the log from `make V=1`, or tracing make/wrapping gcc (but that's unnecessarilly hard)... – Vladislav Ivanishin Jul 02 '19 at 08:21
  • @VladislavIvanishin I have checked optimization flags by using this command: `readelf --debug-dump python_executable | grep --color=always -- '-O' | less -r`. Got many lines with `-Og` option, which is strange, because I were forcing `gcc` to compile with `-O0` only. Then I have changed the `grep` command pattern to the `'-O[^g]'` and got 5 lines with `-O0` options. Thus, the result of this checking is: 5 files have compiled with `-O0`, others with `-Og`. Nothing should interfering to the `GDB` ability to show variable values. – MiniMax Jul 05 '19 at 18:56
  • 1
    @MiniMax, But gcc does optimize with `-Og`. See for example [this question](https://stackoverflow.com/q/31435771/2104472). – Vladislav Ivanishin Jul 05 '19 at 20:19
  • 1
    @VladislavIvanishin You were right, the reason is '-0g' option. Thanks for advise. I don't understand what I did, but after multiple trying to compile `python` executable by using different options for `./configure` and removing the old `Makefile` every time, it created the right **ELF** file with `-O0` for all source files. Not a single `-0g` file. I had been tracing the result by `readelf --debug-dump python | sed -n '/-O/{h;n;n;G;s/.*/&\n/p}' | less` command after each compilation - it shows `-O` option and source filename. Now it works as it should in `GDB` - without "optimized out". – MiniMax Jul 05 '19 at 23:24

0 Answers0