How to retain existing env variables in a new shell

Question

I know I must be doing something silly here, but I'm trying to pass environment variables to a command being run under /bin/sh -c in Cloud Build.

My Cloud Build file looks like this:

 - id: db-migrate
  name: node:16.13.0
  dir: 'packages/backend'
  entrypoint: '/bin/sh'
  args: 
 - '-c'
 - '(/workspace/cloud_sql_proxy -dir=/workspace -instances=$_DB_CONNECTION=tcp:127.0.0.1:5432 & sleep 2) && yarn db:migrate'
  env:
 - 'DB_HOST=127.0.0.1'
 - 'DB_USER=$_DB_USER'
 - 'DB_PASSWORD=$_DB_PASSWORD'
 - 'DB_NAME=$_DATABASE'

My Cloud Build Trigger has the substitutions set, and when I look at the build details it shows the environment variables as set.

However the command yarn db:migrate acts as if there are no env variables set. I believe this is because they aren't being passed from the machine to the command.

Any idea what I'm doing wrong?

The problem here is that when we call bin/sh it's creating a new shell with it's own environment variables. While I read through the manual on SH/Dash I will leave this question here: How do I retain existing env variables in a new shell?

try wrapping in " " instead of ' ', the former in bash scripts translate $ to literal strings — Christopher Hoffman, Jan 11 '22 at 01:17
@ChristopherHoffman These are YAML quotes, not shell quotes. But that raises the question, do `$_DB_USER`, `$_DB_PASSWORD` and `$_DATABASE` get expanded by Cloud Build? They aren't shell arguments, so there's no reason to believe they will be expanded as shell variables. — chepner, Jan 11 '22 at 14:02
@chepner There is probably an easier way, then i would build the YAML file using shell script, that will expand the environment variables — Christopher Hoffman, Jan 11 '22 at 14:40
The real question is, where do `_DB_USER` et al. get defined in the first place? You might be able to use YAML references, or you might consider using Secret Manager, or generate the Cloud Build file dynamically, or some other option. — chepner, Jan 11 '22 at 14:45
Sorry all! I wasn't receiving notifications on this. In regards to your question @chepner I am using Substitution to set the env variables for this step. Regarding where it get's defined is in the trigger. I am going to be using secret manage once I get this working. I think I found the problem and I'm testing it now. Will answer this question once I have finished testing! — Crash, Jan 12 '22 at 15:33

score 0 · Accepted Answer · answered Jan 14 '22 at 07:21

Alright, I figured this out.

We were using TypeORM, and originally used a ormconfig.json file. Turns out, this was still being picked up somehow on the system and was overriding all env variables.

Posting this response to help others in case they make this same mistake.

How to retain existing env variables in a new shell

1 Answers1