Passing parameters between concourse jobs / tasks

Question

What's the best way to pass parameters between concourse tasks and jobs? For example; if my first task generates a unique ID, what would be the best way to pass that ID to the next job or task?

Answer 1

If you are just passing between tasks within the same job, you can use artifacts (https://concourse-ci.org/running-tasks.html#outputs) and if you are passing between jobs, you can use resources (like putting it in git or s3). For example, if you are passing between tasks, you can have a task file

---
platform: linux

image_resource: # ...

outputs:
- name: unique-id

run:
  path: project-src/ci/fill-in-output.sh

And the script fill-in-output.sh will put the file that contains the unique ID into path unique-id/. With that, you can have another task that takes the unique-id output as an input (https://concourse-ci.org/running-tasks.html#inputs) and use that unique id file.

Answer 2

Additionally to tasks resources will place files automagically for you in their working directory.

For example I have a pipeline job as follows

jobs:
- name: build
  plan:
    - get: git-some-repo
    - put: push-some-image
      params:
        build: git-some-repo/the-image
    - task: Use-the-image-details
      config:
        platform: linux
        image_resource:
          type: docker-image
          source:
            repository: alpine
        inputs:
          - name: push-some-image
        run:
          path: sh
          args:
          - -exc
          - |
            ls -lrt push-some-image
            cat push-some-image/repository
            cat push-some-image/digest

Well see the details of the image push from push-some-image

+ cat push-some-image/repository
xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/path/image
+ cat push-some-image/digest
sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Answer 3

Passing data within a job's tasks could easily be done with input/output artifacts (files), As Clara Fu noted.

For the case between jobs, when simple e.g. 'string' data has to be passed , and using a git is an overkill, the 'keyval' resource[1] seems to be a good solution. The readme describes that the data is stored and managed as a standard properties file.

1. https://github.com/SWCE/keyval-resource