Concourse: How to set variables in pipelines file?

Question

So I have been running concourse pipelines using a separate yaml file to hold my variables, similar to this example in the documentation. However I would like to set my variables within my main pipeline.yml file to avoid using the cli option --load-vars-from some_other_file.yml. How could I do this?

Note: I might be looking for something that uses params:, but I want the params I set to be global for everything in my pipeline.yml file, so that everything can use the variables I set in it.

Answer 1

I don't believe that what you want to do is possible, in the way that you suggest.

I think you have two options:

1. Put your YAML file in an S3 bucket, and have the pipeline watch the S3 bucket as a resource, and call set-pipeline on itself whenever that bucket changes, using the YAML file in the bucket to populate variables.
2. Put your YAML file in an S3 bucket, and use it as an input to whatever job needs those variables. You can then use a tool like yml2env to make the contents of that YAML file available to your scripts as environment variables.

Answer 2

As of concourse v3.3.0, you can setup Credential Management in order to use variables from the Vault (this is the only supported credential manager by concourse right now). This way you won't have to keep any variable in a separate file and Vault will keep them secure as well.

Using the Credential Manager you can parameterize:

• source under resources in a pipeline
• source under resource_types in a pipeline
• source under image_resource in a task config
• params in a pipeline
• params in a task config

For setting up vault with concourse you can refer to:

https://concourse-ci.org/creds.html