Linux / Raspberry Pi OS - Systemd not working with python3 script using OS module that accesses enviroment variable

Question

I'm trying to use systemd to run a python3 script, it was working fine, however I changed my python script to use the built in OS module as I wanted to retrieve an enviroment variable from the system to use in the python script as a variable.

The python script is as follows:

#/usr/bin/python

import sys, requests, json, time, os
import paho.mqtt.client as mqtt
from requests.auth import HTTPBasicAuth
from datetime import datetime

MQTT_DEVICE_ID = os.environ['DEVICE_ID']+"/DEFENCE"

The DEVICE_ID enviroment variable comes from one I set in /etc/enviroment:

export DEVICE_ID="TEST1"

user@computer:~ $ echo $DEVICE_ID
TEST1

After adding this change to my python script the systemd service that runs this script no longer starts, it will work for a brief period then it will keep failing and thren working again and failing:

computer@computer:~ $ sudo systemctl status mqtt.service
● mqtt_defense.service - Arms the mqtt.py script that will alert if the device is moved
     Loaded: loaded (/etc/systemd/system/mqtt.service; disabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Thu 2022-08-11 08:57:37 BST; 1s ago
    Process: 2442 ExecStart=python3 /scripts/mqtt.py (code=exited, status=1/FAILURE)
   Main PID: 2442 (code=exited, status=1/FAILURE)
        CPU: 631ms
    
Aug 11 08:57:37 computer systemd[1]: mqtt.service: Main process exited, code=exited, status=1/FAILURE
Aug 11 08:57:37 computer systemd[1]: mqtt.service: Failed with result 'exit-code'.
user@computer:~ $ sudo systemctl status mqtt_defense.service
● mqtt.service - Arms the mqtt.py script that will alert if the device is moved
     Loaded: loaded (/etc/systemd/system/mqtt.service; disabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Thu 2022-08-11 08:57:37 BST; 3s ago
    Process: 2442 ExecStart=python3 /scripts/mqtt.py (code=exited, status=1/FAILURE)
   Main PID: 2442 (code=exited, status=1/FAILURE)
        CPU: 631ms

Aug 11 08:57:37 computer systemd[1]: mqtt.service: Main process exited, code=exited, status=1/FAILURE
Aug 11 08:57:37 computer systemd[1]: mqtt.service: Failed with result 'exit-code'.
user@computer:~ $ sudo systemctl status mqtt.service
● mqtt.service - Arms the mqtt_defense.py script that will alert if the device is moved
     Loaded: loaded (/etc/systemd/system/mqtt.service; disabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-08-11 08:57:59 BST; 304ms ago
   Main PID: 2475 (python3)
      Tasks: 1 (limit: 3720)
        CPU: 295ms
     CGroup: /system.slice/mqtt.service
             └─2475 python3 /scripts/mqtt.py

Aug 11 08:57:59 computer systemd[1]: Started Arms the mqtt.py script that will alert if the device is moved.
user@computer:~ $ sudo systemctl status mqtt.service
● mqtt.service - Arms the mqtt.py script that will alert if the device is moved
     Loaded: loaded (/etc/systemd/system/mqtt.service; disabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Thu 2022-08-11 08:58:40 BST; 2s ago
    Process: 2551 ExecStart=python3 /scripts/mqtt.py (code=exited, status=1/FAILURE)
   Main PID: 2551 (code=exited, status=1/FAILURE)
        CPU: 633ms

After reading a few other questions on this issue such as this, I've tried altering my service to add the WorkingDirectory, User and Group as follows:

[Unit]
Description=Runs the mqtt.py script 
After=multi-user.target

[Service]
WorkingDirectory=/scripts/
User=user
Group=user
Type=simple
ExecStart=python3 /scripts/mqtt.py
Restart=always
RestartSec=5
TimeoutSec=60
RuntimeMaxSec=infinity

[Install]
WantedBy=multi-user.target

I've also tried changing the User and Group to root, still with no success. If I don't use the OS module in my python script the systemd service runs perfectly.

I have a feeling this is a specific problem with the python OS module or that I'm trying to access etc/enviroment in my python script, but I'm not sure what would be the issue.

Any help would be appreciated, thanks.

I’m voting to close this question because From the tag: systemd questions should be for *programming questions* using systemd or its libraries. Questions about *configuring the daemon* (including writing unit files) are better directed to Unix & Linux: https://unix.stackexchange.com. — Rob, Aug 13 '22 at 13:45
This has python code involved in the question, therefore is a valid question for this stack exchange site. So I'm not going to close it. — SneakyShrike, Aug 13 '22 at 14:20

score 1 · Accepted Answer · answered Aug 12 '22 at 02:29

1

This has nothing to do with Python. Your problem is that systemd doesn't expose the env vars defined in /etc/environment by default. See, for example, https://unix.stackexchange.com/questions/473001/env-vars-in-etc-environment-not-globally-visible. As explained in the answer to that question /etc/environment is only loaded by PAM (pluggable authentication module) managed sessions such as interactive logins. AFAIK, systemd doesn't use PAM.

answered Aug 12 '22 at 02:29

Kurtis Rader

6,734
13
20

I’ve had a look at the question you linked and tried the following: Within my service I’ve tried to add EnvironmentFile=/etc/environment which contains my environment variable: export DEVICE_ID="TEST1". I’ve also tried using the systemctl edit mqtt.service to create mqtt.service.d/override.conf file that also has a EnvironmentFile=/etc/environment set. Next I tried instead to set the Environment=$DEVICE_ID in the mqtt.service directly as well as the mqtt.service.d/override.conf file. – SneakyShrike Aug 12 '22 at 13:09
Finally I tried all of the above but changing where the DEVICE_ID environment variable was set. So far I’ve tried setting it in etc/profile with a value of export DEVICE_ID="TEST1". I also tried creating a device_id.sh file within /etc/profile.d/device_id.sh which has a value of export DEVICE_ID=TEST1. Unfortunalty I get the same result where the service fails to start as I showed in my prior question. Prehaps I’m missing something? – SneakyShrike Aug 12 '22 at 13:09
@SneakyShrike: Yes, you are missing something. :-) Start by noticing that */etc/profile* is read by _login_ shells. Which means it is never read by non-interactive shells. Read https://www.freedesktop.org/software/systemd/man/systemd.exec.html. Then double check that your */etc/environment* contains a line like `DEVICE_ID=some_value`. – Kurtis Rader Aug 13 '22 at 02:56
It turns out my issue was that in the /etc/environment I used export which isn't allowed apparently. – SneakyShrike Aug 14 '22 at 10:58

Linux / Raspberry Pi OS - Systemd not working with python3 script using OS module that accesses enviroment variable

1 Answers1

Linked