Error importing python module from C++: ModuleNotFoundError: No module named '.'

Question

I am trying to import a python module and execute a function from it in C++. I have the following snippet:

int python_exec(const fs::path& module_path, std::string_view function) {
    const auto tmp = module_path.string();

    PyObject* module = PyImport_ImportModule(tmp.data());

    if (!module) {
        return -1;
    }

    PyObject* symbol = PyObject_GetAttrString(module, function.data());

    if (!symbol) {
        return - 1;
    }

    if (PyCallable_Check(symbol)) {
        PyObject* value = PyObject_CallObject(symbol, nullptr);
        return PyLong_AsLong(value);
    }

    return -1;
}

int main(int argc, char** argv)
{
    const auto current = fs::current_path();
    const auto module  = argv[1];

    std::cout << current << '\n';        // "C:\\full\\path\\to\\project\\cmake-build-debug"
    std::cout << module  << '\n';        // "../test/test.py"


    Py_Initialize();
    Py_SetPath(current.c_str());

    std::cout << python_exec(module, "my_func") << '\n';

    Py_Finalize();

    return 0;
}

and the following project structure:

project
|_cmake-build-debug
|    |_app.exe
|_test
     |_test.py

My executable is running in the build directory and I pass "../test/test.py" as a program argument

Python throws a ModuleNotFoundError: No module named '.' error.

I also tried calling Py_SetPath before Py_Initialize as suggested here https://stackoverflow.com/a/63452021/22124714 but this results in other errors that I can't fully interpret:

Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = 'C:\\full\\path\\to\\project\\cmake-build-debug\\pytest.exe'
  sys.base_prefix = ''
  sys.base_exec_prefix = ''
  sys.platlibdir = 'lib'
  sys.executable = 'C:\\full\\path\\to\\project\\cmake-build-debug\\pytest.exe'
  sys.prefix = ''
  sys.exec_prefix = ''
  sys.path = [
    'C:\\full\\path\\to\\project\\cmake-build-debug',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Those two lines seems suspicious though.

PYTHONHOME = (not set)
PYTHONPATH = (not set)

I guess I'm not understanding the Py_SetPath function correctly, how can I fix this ?

user22124714 · Accepted Answer · 2023-06-26T16:27:14.157

The issue was about the paths Python use to look for modules, aka sys.path

There are multiple ways to set the path:

PySys_SetPath

Py_Initialize();
PySys_SetPath(L"../test")

PyList_Append

void Py_AppendPath(std::string_view module_dir)
{
    PyObject* msys = PyImport_ImportModule("sys");
    PyObject* path = PyObject_GetAttrString(msys, "path");

    PyList_Append(path, PyUnicode_FromString(module_dir.data()));
}

Py_Initialize();
Py_AppendPath("../test");

PyRun_SimpleString()

const fs::path moddir = fs::absolute(argv[1]).remove_filename();
const std::string ex = std::format(R"(sys.path.append(r"{}\"))", moddir.string());
PyRun_SimpleString("import sys");
PyRun_SimpleString(ex.data());

Note:

Avoid naming your module "test" because there is already one existing
PyImport_ImportModule only takes the module's name, omit the file extension.
Code does not have error checking and reference counting to make it clearer. Make sure to handle those !

Error importing python module from C++: ModuleNotFoundError: No module named '.'

1 Answers1