3

My problem is pretty simple, I have a shared library compiled with CMake and I want to write a python wrapper for one cpp class it contains and a standalone function.

Let's say the class is this with the standalone function

// cppClass.cpp
void standalone() 
{ 
    printf("I am not alone\n"); 
}
int cppClass::not_static_method(int value)
{
    printf("I am not static\n");
    return value;
}
cppClass* cppClass::static_method()
{
    cppClass* c = new cppClass();
    if (!c) { return NULL; }
    return c;
}

// cppClass.h
#ifndef __CPPCLASS_H__
#define __CPPCLASS_H__
void standalone();
class cppClass
{
public:
    int not_static_method(int value);
    static cppClass* static_method();
};
#endif

So I declare it in my .pxd file and write a little wrapper class in my .pyx file:

# .pxd
cdef extern:
    cdef cppclass cppClass:
        int not_static_method(int value)
        @staticmethod
        cppClass* static_method()
    cdef void standalone()

# .pyx
cdef class PyClass:
    cdef cppClass* c_clazz
    def __cinit__(self):
        self.c_clazz = cppClass.static_method()
    def NotStatic(self, value):
        standalone()
        return self.c_clazz.not_static_method(value)

Problem is, once compiled, I can initialize a PyClass object but calling the method NotStatic of the latter object raises an undefined symbol: standalone and when I comment the call to this function, it raises a segmentation fault which I think is due to the initialization of the pointer c_clazz inside the PyClass object.

I understood that I cannot specify where the definitions are since it is an extern library, and I already specified its name in the setup.py file as well as its path as extra_link_args.

What am I doing (possibly extremely) wrong?

Edit: shared library is compiled with g++ -shared -fPIC -c cppClass.cpp -o libcppClass.so

Edit 2: added .h file

Edit 3: I did not mention why I want to use a shared library, its because the non static method uses some definition in CUDA files compiled into the shared library as well.

Edit 4: my setup file

# setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
    cmdclass = {"build_ext": build_ext},
    ext_modules = [
            Extension("cy_wrapper",
                      sources=["cy_wrapper.pyx"],
                      libraries=["mycustomlib"],
                      language="c++",
                      extra_compile_args=["-O3", "-Wall", "-std=c++11"],
                      extra_link_args=["-L/absolute/path/to/libmycustomlib.so/directory/"]
                      )
                  ]
)
Romzie
  • 457
  • 4
  • 11
  • Didn't you forget to add the `int` parameter to `static_method`? – UnholySheep Mar 21 '18 at 08:20
  • Just forgot to remove it from the cpp file I copied sorry, besides it would not compile if I had messed up with the arguments. – Romzie Mar 21 '18 at 08:38
  • 3
    I think you need to use the header `cppClass.h` in your extern block? (probably also adding its path to `include_dirs`) – chrisb Mar 21 '18 at 18:20
  • Your setup looks kind of ok. You need to analize what goes wrong. First verify that your build really worked: Add "-Wl,--no-undefined" to link-args: Right now the linker doesn't care if all symbols are found or not so you see missing values only during the run time. You can also add "-Wl,--verbose" to see which libraries are used during the linkage. Call `readelf -d cy_wrapper.so| grep NEEDED` to see whether `mycustomlib` is really marked as needed. Use `LD_DEBUG=libs,files,symbols python -c "import cy_wrapper"` to see what goes wrong during the run time. – ead Mar 22 '18 at 07:50
  • @ead You're right, `mycustomlib` is not marked as needed and adding `-Wl,--no-undefined` arg shows a lot of undefined reference of `PySomeFunc` AND my `standalone` func AND my `static` method. I don't get it for the static one, I can call it in python and see the c++ output. @chrisb I wanna avoid using this since I am using a shared lib which I'm not supposed to have the source code. – Romzie Mar 22 '18 at 08:33
  • What is your link command? The `PySomeFunc`s isn't a problem, they will be provided by Python-interpreter, but not the others. Run the linker with `-Wl,--verbose` to why your so-Library isn't found. – ead Mar 22 '18 at 09:01
  • `aarch64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-aarch64-3.5/SO.o -lcppClass -o /home/path/to/my/dir/SO.cpython-35m-aarch64-linux-gnu.so -L/home/path/to/my/dir -Wl,--verbose` It is really strange, all libraries are eventually found and opened when I run the command – Romzie Mar 23 '18 at 08:52

1 Answers1

2

Inspecting the code generated by Cython (cy_wrapper.cpp) shows that function standalone is declared as

__PYX_EXTERN_C DL_IMPORT(void) standalone(void);

i.e. 

extern "C" void standalone(void);

which is not as defined in cppClass.h (C++ name mangling issue).

Also cppClass is somewhat different in the generated file

struct cppClass;
struct cppClass {
    virtual int not_static_method(int);
    static cppClass *static_method(void);
    virtual ~cppClass() { }
};

i.e. not as defined in cppClass.h. The definitions should match or there can be trouble like segmentation faults.

I would recommend including the header file in .pxd file i.e.

cdef extern from "cppClass.h"
J.J. Hakala
  • 6,136
  • 6
  • 27
  • 61