Python C API call to error() binds to libc implementation instead of a local one

Question

EDITTED

See at end of post for edit in response to Employed Russian's comment

Disclaimer 

Before going forward, I know that naming a function error is generally bad practice since it may clash with a similar function in libc, but this is an issue I have with some third-party software on which I have little control. Plus I would really like to understand where this error comes from :-)

The issue

The problem I have is that the code below, when executed through the Python interpreter instead of calling my local implementation of the error function, is actually calling the libC's error function instead (as shown by GDB's stack trace below).

When simply compiling the same code within another C program, I do not have such issues. Does someone knows where that comes from ? Does it have to do with the way Python loads shared libraries ?

MCVE

#include <stdio.h>
#include <Python.h>

static PyObject* call_error(PyObject *self, PyObject *args);
static PyMethodDef module_methods[] = {
     {"error", call_error, METH_NOARGS, "call error"},
     {NULL, NULL, 0, NULL}
};

static struct PyModuleDef module_defs = {
     PyModuleDef_HEAD_INIT,
     "Test", "Test", -1, module_methods, NULL, NULL, NULL, NULL};

PyObject* PyInit_Test(void)
{
     PyObject *module = PyModule_Create(&module_defs);
     return module;
}

void error(const char* fmt, ...);

PyObject* call_error(PyObject *self, PyObject *args)
{
     error("Error!");
     Py_RETURN_NONE;
}

void error(const char* fmt, ...)
{
     va_list ap;
     va_start(ap, fmt);
     vprintf(fmt, ap);
     va_end(ap);
}

GDB output

Here is the output of importing running the above code within GDB using python3 -c "import Test; Test.error()"

GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3...(no debugging symbols found)...done.
(gdb) r -c 'import Test; Test.error()'
Starting program: /usr/bin/python3 -c 'import Test; Test.error()'
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
/usr/bin/python3:
Program received signal SIGSEGV, Segmentation fault.
__strchrnul_sse2 () at ../sysdeps/x86_64/multiarch/../strchr.S:32
32  ../sysdeps/x86_64/multiarch/../strchr.S: No such file or directory.
(gdb) where
#0  __strchrnul_sse2 () at ../sysdeps/x86_64/multiarch/../strchr.S:32
#1  0x00007ffff6c2c432 in __find_specmb (format=0x4 <error: Cannot access memory at
#     address 0x4>) at printf-parse.h:108
#2  _IO_vfprintf_internal (s=0x7fffffffae60, format=0x4 <error: Cannot access memory at 
#     address 0x4>, ap=0x7fffffffd5b0) at vfprintf.c:1320
#3  0x00007ffff6c2f680 in buffered_vfprintf (s=s@entry=0x7ffff6fbd680 <_IO_2_1_stderr_>,
#     format=format@entry=0x4 <error: Cannot access memory at address 0x4>,
#     args=args@entry=0x7fffffffd5b0) at vfprintf.c:2329
#4  0x00007ffff6c2c726 in _IO_vfprintf_internal (s=0x7ffff6fbd680 <_IO_2_1_stderr_>,
#     format=format@entry=0x4 <error: Cannot access memory at address 0x4>, 
#     ap=ap@entry=0x7fffffffd5b0) at vfprintf.c:1301
#5  0x00007ffff6cef9bb in error_tail (status=status@entry=-161613509, 
#     errnum=errnum@entry=0, message=message@entry=0x4 <error: Cannot access memory at 
#     address 0x4>, args=args@entry=0x7fffffffd5b0) at error.c:271
#6  0x00007ffff6cefb3d in __error (status=-161613509, errnum=0, message=0x4 
#     <error: Cannot access memory at address 0x4>) at error.c:321
#7  0x00007ffff65df82e in call_error (self=0x7ffff67f3548, args=0x0) at test.c:24
#8  0x00000000004c5352 in _PyCFunction_FastCallKeywords ()
#9  0x000000000054ffe4 in ?? ()
#10 0x00000000005546cf in _PyEval_EvalFrameDefault ()
#11 0x000000000054fbe1 in ?? ()
#12 0x0000000000550b93 in PyEval_EvalCode ()
#13 0x000000000042c4ca in PyRun_SimpleStringFlags ()
#14 0x0000000000441918 in Py_Main ()
#15 0x0000000000421ff4 in main ()

EDIT

I did think about the dlopen issue with importing Python modules and actually the following code compiles and runs just fine and prints out:

> ./main
Hi there

main.c

#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
#include <errno.h>
#include <stdarg.h>

typedef void*(*arbitrary)();

extern void error(const char* fmt, ...);

int main(int argc, char **argv)
{
     void *handle;
     arbitrary my_function;

     handle = dlopen("./libtest.so", RTLD_LAZY | RTLD_GLOBAL);
     if (!handle) {
      fprintf(stderr, "%s\n", dlerror());
      exit(EXIT_FAILURE);
     }

     dlerror();    /* Clear any existing error */

     *(void**)(&my_function) = dlsym(handle,"foo");
     (void) my_function();

     // Note: binding using dlsym(handle, "error") works too

     dlclose(handle);
     exit(EXIT_SUCCESS);
}

test.c

#include <stdio.h>
#include <stdarg.h>

extern void error(const char* fmt, ...);
extern void foo(void);

void foo(void)
{
     error("Hi there\n");
}

void error(const char* fmt, ...)
{
     va_list ap;
     va_start(ap, fmt);
     vprintf(fmt, ap);
     va_end(ap);
}

Answer 1

this is an issue I have with some third-party software on which I have little control.

If you have sources for this third_party software, you can edit them, or use macro tricks to rename the function, e.g. -Derror=foo_error.

If you only have an archive library, use objcopy --redefine-symbol ....

If you only have a shared library, I don't know of a workable solution.

Does it have to do with the way Python loads shared libraries ?

Kind of. What's happening is that the dynamic loader resolves a reference to error to the earliest exported definition of that function.

When you link error into your main a.out, that definition is the first in linker search order, so it "wins".

When you use dlopen to load libfoo.so which contains error (which is what Python does for import), that library is loaded after libc.so.6, which means that libc.so.6 appears earlier in the loader search order, and its definition "wins".

You don't need Python to see this: write a trivial test that uses dlopen, and the same problem will show up in it.

Update:

I had written a small test case

Your test case does confirm my answer. You probably didn't build it correctly.

$ gcc -fPIC -shared -o libtest.so test.c
$ gcc main.c -ldl 

Here the "wrong" error is called because the order of library loading is: a.out, libc.so.6, then libtest.so:

$ ./a.out
./a.out: UH��H�=�: Unknown error 640192728

But what you probably did was this:

$ gcc main.c ./libtest.so -ldl

Here the order of library loading is a.out, libtest.so (because a.out directly depends on libtest.so), then libc.so.6, and the "right" error gets called:

$ ./a.out
Hi there