0

I am using xarray with cfgrib to load grib files in Python. I have custom grib definitions, which I am providing to eccodes (backend for cfgrib) via the environment variable GRIB_DEFINITION_PATH. This setup works well, as long as I run the Python script in an environment where the variable was already set.

Now I want to be more flexible with my setup and provide the environment variable from within Python using os.environ (see the example below). But somehow when setting up the environment like this, the variable gets ignored and I don't understand why. Can anyone provide me some insight into this mystery? Thanks in advance!

Here an "MRE" of the setting.

import xarray as xr
import os

grib_definitions_path = "/paths/to/definitions:/split/like/this"
os.environ["GRIB_DEFINITION_PATH"] = grib_definitions_path

grib_file = '/path/to/grib/file'
backend_args = {
    "filter_by_keys": {"shortName": "P"}
}
array = xr.open_dataset(grib_file, engine="cfgrib", encode_cf=("geography", "vertical"), backend_kwargs=backend_args)["P"]
print(array.dims)

Executing the above code in a terminal fails for me with KeyError: 'P'. If I however first run

export GRIB_DEFINITION_PATH="/paths/to/definitions:/split/like/this"

the dimensions of array are being printed as expected.

tierriminator
  • 587
  • 5
  • 19
  • 1
    Try setting the environment before you `import xarray` – Barmar Oct 03 '22 at 20:44
  • How in the world?... I already tried that but it didn't work earlier. But now it somehow seems to work. Thanks a lot! – tierriminator Oct 03 '22 at 20:47
  • You still need to run that line after `os.environ["GRIB_DEFINITION_PATH"] = grib_definitions_path` Add the following `subprocess.check_call(['sqsub', '-np', sys.argv[1], grib_definitions_path ],env=dict(os.environ, SQSUB_VAR="visible in this subprocess"))` – blackbird Oct 04 '22 at 07:57
  • @blackbird Can you explain why? I don't have `sqsub` and I also don't pass any arguments to my run script, hence `sys.argv[1]` would be undefined. – tierriminator Oct 10 '22 at 15:37

0 Answers0