Problem creating OpenMP enabled PCH programmatically with cppyy

Question

I've been trying to generate a OpenMP enabled precompiled header (PCH) with cppyy and have been failing so far. Instead of doing so manually each time as outlined in this answer, I'm looking for an automated solution.

So far, the broad method is to use os.environ to export environmental variables - but this seems to fail somewhat inconsistently, resulting in single-threaded code each time.

In particular, there doesn't seem to be any output from cppyy_backend.loader.ensure_precompiled_header all the time. Even if there is output, it has no effect on whether the C++ code runs parallely or not.

The relevant part of the code below:

import cppyy_backend.loader as l 

os.environ['EXTRA_CLING_ARGS'] = '-fopenmp -O2 -g'
l.set_cling_compile_options(True)
current_folder = os.path.split(os.path.abspath(__file__))[0]
pch_folder = os.path.join(current_folder, 'cling_pch/')
if os.path.exists(pch_folder):
    pass
else:
    os.mkdir(pch_folder)

l.ensure_precompiled_header(pch_folder)
    
# find PCH file:
pch_path = glob.glob(pch_folder+'/allDict*')
if len(pch_path)>0:
    os.environ['CLING_STANDARD_PCH'] = pch_path[0]
else:
    raise ValueError('Unable to find a precompiled header...')

System specs:

Ubuntu 18.04.2 LTS
python 3.9.0
cppyy 2.4.0
cppyy_backend 6.27.0

EDIT: I made sure the custom PCH folder was first empty while checking to see if the custom PCH file was being created.

score 1 · Accepted Answer · answered Oct 19 '22 at 21:57

If you set CLING_STANDARD_PCH to the desired name, it will be used as the name for the generated PCH. Conversely, if you set the folder name to the lower level call from loader, it will set CLING_STANDARD_PCH for you. Setting both should not be necessary.

The default naming scheme will generate a differently named PCH if OpenMP is enabled from when it's not (a .omp. part is added to the name). This allows you to go back end forth, without bothering having to rebuild the PCH each time. Likely, you have multiple editions in the cling_pch directory, and pch_path[0] may well be a non-OpenMP variant.

If you want to stay close to the above, though, then how about:

import os
  
os.environ['EXTRA_CLING_ARGS'] = '-fopenmp -O2 -g'
current_folder = os.path.split(os.path.abspath(__file__))[0]
pch_folder = os.path.join(current_folder, 'cling_pch')
if os.path.exists(pch_folder):
    pass
else:
    os.mkdir(pch_folder)

os.environ['CLING_STANDARD_PCH'] = os.path.join(pch_folder, 'std_with_openmp.pch')

import cppyy

cppyy.cppexec("std::cerr << _OPENMP << std::endl")

_/\\_. It is now working fine - was wondering if there's scope to add more details to the docs (would be happy to contribute). — Thejasvi, Oct 20 '22 at 16:01
The deal is that for the longest while I've been wanting to move to pre-compiled modules, at which point Clang is supposed to take care of all the messy details such as compiler flags (haven't tested that, so will have to see). The holdup is Windows, but I've finally started to make it a Linux optional feature to get going. — Wim Lavrijsen, Oct 20 '22 at 16:19

score 0 · Answer 2 · answered Oct 21 '22 at 04:02

I'm not sure about this, and will be able to confirm in a few days, but... I now suspect the reason there was an inconsistency in PCH creation was that cppyy was being called before the EXTRA_CLING_ARGS were being set.

I guess this means the 'standard' pre-compiled header was first being loaded and the 'custom' arguments were being ignored. During debugging or over the course of multiple runs - sometimes the effective order of execution was reversed.

Problem creating OpenMP enabled PCH programmatically with cppyy

2 Answers2