In Concourse, can I download a file from a container that was used during the pipeline run?

Question

One of the steps in my Concourse pipeline runs tests. The tests are failing, and I'd like to look at a file that is created by the test. I can fly hijack the container, and can see the file. But I need to download the file to my laptop (it's a Cypress MP4 file). Is there some way I can manually download that file from the container? Or do I have to upload it somewhere as part of the pipeline?

Answer 1

At first, prepare tar.gz file in the failed container.

fly hijack -t your-team -b <build-num> -s <step-name> -j pipeline-name/job-name sh
$ tar zcvf cypress.tar.gz cypress
$ exit

Then transfer tar.gz file from the container to local.

fly hijack -t your-team -b <build-num> -s <step-name> -j pipeline-name/job-name cat /tmp/build/123456ab/cypress.tar.gz > ./cypress.tar.gz

Answer 2

yes, you have to upload the file which you want to see. Upload in s3 (you can find other resources also) via concourse. For uploading a file into s3, you can configure s3 resource in concourse like this:

name: s3_upload_file
  type: s3
  source:
    access_key_id: <give_access_key>
    bucket: <bucket-name>
    secret_access_key: <secret-access-key>
    regexp: <file name or you can specify regex also here>

After uploading a file into s3, put an s3 URL in a file via concourse script like following:

echo -e "File can be downloaded from <s3-url> >> slack.txt"

And then sent this message into slack like this:

- put: slack-alert
        params:
          always_notify: true
          channel: ((slack-channel-name))
          text_file: <slack.txt file location>
          text: |
            $TEXT_FILE_CONTENT

So, with the URL anyone can download a file from s3 or any other source wherever you have uploaded.

Answer 3

Transferring the file via fly hijack is not possible as far as I know. I guess you could upload the file to some remote from the container itself if you got the respective cli tools (curl, scp, ..). As an alternative, you can upload the file from within the pipeline, as mentioned in the answer of "Anshita Singh".