How to substitute values in a key-value file if keys exists in shell environment?

Question

I have a file like below with key value pair separated by =. I have already exported some variables to my shell. Now I want a shell script which can substitute the values for its corresponding keys if those keys are already exist in env variables or exported to shell.

I can have used envsubst but I didn't want to prefix a dollar symbol with those default values. If there is any other tool like envsubst which can help me substitute the string value please refer that also.

foo.yaml

PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network    
ENVIRONMENT=dev
DRUPAL_VERSION=8
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=pass
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass

I don't know why It is getting negative vote. Isn't my question clear or is a very basic question ? — SkyRar, Dec 14 '18 at 00:28

stack0114106 · Accepted Answer · 2018-12-15T02:36:07.783

I think, you are looking for something like below.

> cat foo.yaml
PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network
ENVIRONMENT=dev
DRUPAL_VERSION=8
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=pass
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass
>
> export ENVIRONMENT=sit
> export DRUPAL_VERSION=10
> export MYSQL_PASSWORD="*****"
>
> perl -lpe ' s/(.*)=(.*)/sprintf("%s=%s","$1",$ENV{$1}? $ENV{$1}:$2)/ge ' foo.yaml
PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network
ENVIRONMENT=sit
DRUPAL_VERSION=10
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=*****
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass
>

Does this help?.

Note that you can use -i switch in Perl to overwrite the file if you have write permissions

> perl -i -lpe ' s/(.*)=(.*)/sprintf("%s=%s","$1",$ENV{$1}? $ENV{$1}:$2)/ge ' foo.yaml
> cat foo.yaml
PROJECT_NAME=test
DOMAIN_NAME=test.localhost
NETWORK_NAME=traefik-network
ENVIRONMENT=sit
DRUPAL_VERSION=10
MYSQL_HOSTNAME=test.mariadb
MYSQL_DATABASE=drupal
MYSQL_USER=test
MYSQL_PASSWORD=*****
MYSQL_PORT=3306
MYSQL_ROOT_PASSWORD=pass
>

AWK solultion.

> awk -F"=" ' { $2=ENVIRON[$1]?ENVIRON[$1]:$2; printf("%s=%s\n",$1,$2) } ' foo.yaml

can be further shortened as

> awk -F"=" ' { printf("%s=%s\n",$1,ENVIRON[$1]?ENVIRON[$1]:$2) } ' foo.yaml

Handling blank lines

> awk -F"=" ' { if (!/^\s*$/) $0=sprintf("%s=%s",$1,ENVIRON[$1]?ENVIRON[$1]:$2) }1 ' foo.yaml

Thanks for taking the efforts to help me. I wanted inplace edit that your second answer solves. Can you do the same in awk and sed also? — SkyRar, Dec 14 '18 at 11:21
added awk solution.. redirect it to another file and rename to original file..pls check — stack0114106, Dec 14 '18 at 11:38
I will surely accept it as answer. How ever the awk one adds one `=` in blank line — SkyRar, Dec 14 '18 at 20:16
the blank line issue with awk has been fixed.. check the updated answer. — stack0114106, Dec 15 '18 at 02:36
Thank you. awk solution is still needs some changing. It postpend an equal symbol on comment line. like `# General config=` — SkyRar, Dec 15 '18 at 21:22

How to substitute values in a key-value file if keys exists in shell environment?

1 Answers1