3

I am facing a weird problem when compiling a Python extension featuring OpenMP with Clang.

Minimal Example

I managed to boil down my actual problem to the following code: The Python extension could not be much simpler, while still featuring OpenMP. Apart from the function bar, this is mostly standard boilerplate:

# include <Python.h>

static PyObject * bar(PyObject *self)
{
    #pragma omp parallel sections
    {
        #pragma omp section
        {float x=42.0; x+=1;}
    }
    Py_RETURN_NONE;
}

static PyMethodDef foo_methods[] = {
    {"bar", (PyCFunction) bar, METH_NOARGS, NULL},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef moduledef = {
    PyModuleDef_HEAD_INIT, "foo", NULL, -1,
    foo_methods, NULL, NULL, NULL, NULL
};

PyMODINIT_FUNC PyInit_foo(void)
{
    return PyModule_Create(&moduledef);
}

With the above being foo.c, I compile and load this with:

clang -fPIC -fopenmp -I/usr/include/python3.7m -c foo.c -o foo.o
clang -shared foo.o -o foo.so -lgomp
python3 -c "import foo"

The last line, i.e., the import of the module throws the following error:

ImportError: /home/wrzlprmft/…/foo.so: undefined symbol: __kmpc_for_static_fini

What I found out so far

  • This does not happen when replacing Clang with GCC.
  • This does not happen with regular shared libraries (not involving Python).
  • Using Setuptools to compile the extension does not help. (In fact, my compile commands are a reduction of what Setuptools does to find out whether it uses any non-essential compiler extensions that cause this.)
  • All of this happens on Ubuntu 19.10 with Python 3.7, Clang 9.0.0-2, and GCC 9.2.1.
  • I can also replicate the problem on current Arch Linux with Python 3.8 and Clang 9.0.1.
  • This worked until a year ago, probably longer.
  • Using Python 3.6 does not help.
  • Using Clang 3.8, 4.0, 6.0, 7, or 8 does not help.
  • Here, somebody reports a similar problem when trying to compile TensorFlow. This is yet unsolved.

Question

What is going wrong here and how can I fix this? Right now I do not even have an idea whether this is an error by me, in Clang, OpenMP, or Python.

Wrzlprmft
  • 4,234
  • 1
  • 28
  • 54
  • 1
    is that helping: https://stackoverflow.com/questions/13715979/parallel-program-giving-error-undefined-reference-to-kmpc-ok-to-fork ? – Mr Cas Mar 06 '20 at 16:11
  • @MrCas: Kind of. Adding `-fopenmp` to the second compilation step solves the problem for the minimal example, but only when using Clang as a compiler there. Unfortunately it turns out that Setuptools doesn’t respect my compiler choice here, but that’s a different problem. (Feel free to expand your comment to an answer.) – Wrzlprmft Mar 06 '20 at 16:31
  • Using -lgomp definitely won't provide the runtime functions clang requires. (Though the converse, using CLANG's libomp.so instead of libgomp.so should for GCC compiled code!) – Jim Cownie Mar 09 '20 at 08:46
  • there is no need to add `-lgomp` if `-fopenmp` is added - the frontend (gcc or clang) will pick up the right dependency automatically. – ead Mar 09 '20 at 10:59

0 Answers0