Python Script Running in Virtual Environment on RPi Cannot Find AWS Environment Variables

Question

I have a Python script that runs in a virtual environment on boot from a service file on a Raspberry Pi. The service file is shown below:

[Unit]
Description=Starts IEQ sensors

[Service]
ExecStart = /home/pi/bevo_iaq/.venv/bin/python3 -E /home/pi/bevo_iaq/bevobeacon-iaq/main.py
Restart=always
RestartSec=60s

[Install]
WantedBy=bevobeacon.service

The main.py script includes the following lines to access environment variables:

# AWS credentials
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']
BUCKET_NAME = os.environ['BUCKET_NAME']

I export these variables as the last line from /etc/profile:

export AWS_ACCESS_KEY_ID='<aws_access_key>'
export AWS_SECRET_ACCESS_KEY='<aws_secret_access_key>'
export BUCKET_NAME='<bucket_name>'

When the RPi tries to run the script on boot via the service file, I get the following error:

Feb 21 08:43:57 raspberrypi systemd[1]: Started Starts IEQ sensors.
Feb 21 08:43:58 raspberrypi sudo[296]:     root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/home/pi/bevo_iaq/.venv/bin/python3 -E /home/pi/be
Feb 21 08:43:58 raspberrypi sudo[296]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 21 08:44:03 raspberrypi sudo[296]: environ({'LANG': 'en_GB.UTF-8', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
Feb 21 08:44:03 raspberrypi sudo[296]: Traceback (most recent call last):
Feb 21 08:44:03 raspberrypi sudo[296]:   File "/home/pi/bevo_iaq/bevobeacon-iaq/main.py", line 213, in <module>
Feb 21 08:44:03 raspberrypi sudo[296]:     asyncio.run(main(beacon = beacon))
Feb 21 08:44:03 raspberrypi sudo[296]:   File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run
Feb 21 08:44:03 raspberrypi sudo[296]:     return loop.run_until_complete(main)
Feb 21 08:44:03 raspberrypi sudo[296]:   File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
Feb 21 08:44:03 raspberrypi sudo[296]:     return future.result()
Feb 21 08:44:03 raspberrypi sudo[296]:   File "/home/pi/bevo_iaq/bevobeacon-iaq/main.py", line 32, in main
Feb 21 08:44:03 raspberrypi sudo[296]:     AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
Feb 21 08:44:03 raspberrypi sudo[296]:   File "/usr/lib/python3.7/os.py", line 678, in __getitem__
Feb 21 08:44:03 raspberrypi sudo[296]:     raise KeyError(key) from None
Feb 21 08:44:03 raspberrypi sudo[296]: KeyError: 'AWS_ACCESS_KEY_ID'

However, when I run the script manually, it works with no issues:

pi@raspberrypi:~/bevo_iaq $ source .venv/bin/activate
(.venv) pi@raspberrypi:~/bevo_iaq $ python3 -E bevobeacon-iaq/main.py 
.
.
.
/home/pi/DATA/b07_2022-02-21.csv was uploaded to  AWS S3 bucket: <bucket_name>

I am not sure what the issue is but it seems like the environment variables are not loaded in correctly when the script runs from boot. I saw from another question that the environment variables are cleared when using sudo which I am not, but I also see from the error log that I am running the script as the root user - perhaps that is the issue?

If you really want to go down the rabbit hole, the entire project can be found here.

Answer 1

The user instance of systemd does not inherit environment variables set in files such as .bashrc, /etc/profile, etc.

There are several ways to set environment variables for a systemd service, one is to provide them in your service file:

[Unit]
Description=Starts IEQ sensors

[Service]
EnvironmentFile=/lib/systemd/system/myenv.env
ExecStart = /home/pi/bevo_iaq/.venv/bin/python3 -E /home/pi/bevo_iaq/bevobeacon-iaq/main.py
Restart=always
RestartSec=60s

[Install]
WantedBy=bevobeacon.service

In myenv.env:

AWS_ACCESS_KEY_ID=SECRET_1
AWS_SECRET_ACCESS_KEY=SECRET_2
BUCKET_NAME=SECRET_3

See: https://unix.stackexchange.com/a/455283