Can a variable expansion in python work like shell expansion of variable

Question

I have a legacy python code that reads from a shell script for variable values like in a properties file. For example, say, I have a shell program x.sh for variable declaration as:

Y_HOME=/utils
Y_SPCL=$Y_HOME/spcl
UTIL1=$Y_SPCL/u1
Y_LIB=$Y_HOME/lib

Now, from within python program abc.py, I read x.sh file line by line and use line.split("=")[1] to get the value of the variable, say, UTIL1 as $Y_SPCL/u1 in non-expanded form and not in expanded form as /utils/spcl/u1. Can I have sone mechanism in python to have vafiable expandion like in a shell program execution. I think, since I am using x.sh not as a shell program, rather as a configuration file like properties, there should be all variables in expanded form to let the python program run properly, such as:

Y_HOME=/utils
Y_SPCL=/utils/spcl
UTIL1=/utils/spcl/u1
Y_LIB=/utils/lib

This will have no change on the legacy python part of code and changing the configuration file as an external properties data. Please pass your opinions.

Why is it easier to implement this in code, rather than just changing the config file to be like your second example? — jordanm, Mar 25 '20 at 20:00
You can have a shell interpret the file, and then query it for resulting variables — that other guy, Mar 25 '20 at 20:06
Is there a good reason why you are parsing the config file yourself in hand-crafted code instead of using the module `configparser`? And whether you do that or not, you can use `os.path.expandvars()` to do what you want. — BoarGules, Mar 25 '20 at 20:13

score 3 · Answer 1 · answered Mar 25 '20 at 20:12

There is a package dotenv that can do that.

pip install --user python-dotenv

Then, in Python:

import dotenv, os
dotenv.load_env("x.sh")
print(os.environ["Y_LIB"])

Important: Make sure your variable substitutions read like ${VAR}. So your x.sh would look like this:

Y_HOME=/utils
Y_SPCL=${Y_HOME}/spcl
UTIL1=${Y_SPCL}/u1
Y_LIB=${Y_HOME}/lib

score 1 · Answer 2 · answered Mar 25 '20 at 20:50

Assuming variables must be declared before use and they form a correct path when expanded, you could do something like this.

There is the file fileName that contains the variables:

Y_HOME=/utils
Y_SPCL=$Y_HOME/spcl
UTIL1=$Y_SPCL/u1
Y_LIB=$Y_HOME/lib

So, For each "variable", you search for it in the next variables "values" and replace it for the proper "value". You could have a .py like this:

variables = []
with open("fileName", 'r') as f:
    while True:
        line = f.readline()[:-1] # Get the line minus '\n'
        if not line:
            break
        variables.append(line.split('=')) 

    for i in range(len(variables)):
        current =  "$" + variables[i][0]
        for j in range(len(variables)):
            replaced = None
            while replaced != variables[j][1]:
                # We replace until no further replaces happen
                replaced = variables[j][1].replace(current, variables[i][1])
                variables[j][1] = replaced

for var in variables:
    print(var[0] + "=" + var[1])

Output:

Y_HOME=/utils
Y_SPCL=/utils/spcl
UTIL1=/utils/spcl/u1
Y_LIB=/utils/li

Can a variable expansion in python work like shell expansion of variable

2 Answers2