Download same pipeline artifact latestFromBranch over multiple jobs

Question

I currently write a Azure pipeline YAML which downloads the latest development artifacts and executes different tests in different environments.

Since every kind of test is modeled as its own Job, the first step of the job is downloading the artifact (documentation).

- task: DownloadPipelineArtifact@2
  inputs:
    buildType: 'specific'
    project: '[hidden]'
    definition: '[hidden]'
    buildVersionToDownload: 'latestFromBranch'
    branchName: 'refs/heads/development'

How can I achieve that every job takes the same artifact, also if a newer is created during the pipeline run?

My first approach was downloading the latest artifact and extracting the concrete version. But I wasn't able to get the build id which is required as pipelineId for downloading a specific version.

Any update for this issue? Have you resolved this issue? If not, would you please let me know the latest information about this issue? — Leo Liu, Oct 11 '19 at 01:24
@LeoLiu-MSFT Sorry, hadn't time so far. I'll leave a comment/vote after I tried your solution. :-) — sschmeck, Oct 11 '19 at 06:10

score 2 · Answer 1 · answered Oct 09 '19 at 10:28

2

But I wasn't able to get the build id which is required as pipelineId for downloading a specific version.

We could use the REST API to get the pipeline build id before we use the task DownloadPipelineArtifact. Add the parameter definitions to get the build id for specify definitions:

https://dev.azure.com/{organization}/{project}/_apis/build/builds?definitions={definitions}&api-version=5.1

Then I got all the build IDs, use Select-Object -first 1 get the latest build ID and set it as environment variable, so my powershell task should be like:

$url = "https://dev.azure.com/{organization}/{project}/_apis/build/builds?definitions={definitions}&api-version=5.1"
$buildPipeline= Invoke-RestMethod -Uri $url -Headers @{   
 Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
} -Method Get

$LatestBuildID= $buildPipeline.value.id | Select-Object -first 1


Write-Host This is Latest Build ID: $LatestBuildID

Write-Host "##vso[task.setvariable variable=LatestBuildID;]$LatestBuildID"

Next, we set the variable $(LatestBuildID) in the Build option:

Now, we could get the latest build id which is required as pipelineId for downloading a specific version.

How can I achieve that every job takes the same artifact, also if a newer is created during the pipeline run?

For this issue, we need set the Build completion for build or Continuous deployment trigger for release, so that there is a new artifact created, the pipeline of DownloadPipelineArtifact should be executed.

Hope this helps.

answered Oct 09 '19 at 10:28

Leo Liu

71,098
10
114
135

Thanks. The URL works in my browser but within the pipeline environment it returns a 'Sign In' page with the message 'Enhanced Security Configuration is currently enabled on your environment. This enhanced level of security prevents our web integration experiences from displaying or performing correctly..' – sschmeck Oct 14 '19 at 08:59
1

@sschmeck, Please try to enable Allow scripts to access the OAuth token in the agent job: https://1drv.ms/u/s!Ai1sp_yvodHf6nQ7C3cUQ3EaYT8d?e=dI3XWx – Leo Liu Oct 14 '19 at 09:10
Couln't find the menu of the screenshot. It turned out that the token was empty. https://stackoverflow.com/a/52838552/846163 fixed the issue. Thanks. – sschmeck Oct 14 '19 at 10:06
@sschmeck, So, the issue in the comment has been resoled, how about the issue in the question? – Leo Liu Oct 16 '19 at 07:07
1

Didn't found a better solution. I followed my approach which works now with your help. :-) But feels more like a workaround than a solution. I'll post my complete solution. Thx. – sschmeck Oct 16 '19 at 07:23
@sschmeck, Great, since I am not in your project, maybe my method is not the best for you. Looking forward to your solution. – Leo Liu Oct 16 '19 at 07:28

sschmeck · Accepted Answer · 2019-10-28T06:57:26.630

I implemented a solution as suggested by @LeoLiu-MSFT.

Find the last successful master build and store the buildId
Provide a download task with the buildId as parameter
Use the download task in all test jobs

The following snippets show a concrete example.

# 1. steps/find-latest-build.yml
steps:
  - task: Bash@3
    name: LatestArtifactsBuild
    displayName: 'Find latest successful master build'
    env:
      system_accesstoken: $(System.AccessToken)
    inputs:
      targetType: 'inline'
      script: |
        definitionId=<pipeline>
        url="https://dev.azure.com/<organization>/<project>/_apis/build/builds?api-version=5.1"
        url+="&definitions=$definitionId"
        url+="&resultFilter=succeeded"
        url+="&branchName=refs/heads/master"
        url+="&\$top=1"

        json=$(curl -sL -H "Authorization: Bearer $system_accesstoken" -H 'Content-Type: application/json' $url)
        buildId="$(echo $json | jq '.value[0].id' )"

        echo "##vso[task.setvariable variable=id;isOutput=true]$buildId"

# 2. steps/download-artifacts-from-pipeline-build.yml
parameters:
  definitionId: <pipeline>
  buildId: $(Build.BuildId)
  artifactName: ''

steps:
  - task: DownloadPipelineArtifact@2
    displayName: 'Download build artifact archives'
    inputs:
      buildType: 'specific'
      project: $(System.TeamProjectId)
      definition: ${{ parameters.definitionId }}
      buildVersionToDownload: 'specific'
      pipelineId: ${{ parameters.buildId }}
      artifactName: ${{ parameters.artifactName }}

# 3. examples/reuse-build-artifacts.yml
trigger: none

jobs:
  - job: PrepareBuild
    displayName: 'Provide build information'
    steps:
    - checkout: none
    - template: ../steps/find-latest-build.yml

  - job: DownloadAllArchivesOfSpecificBuild
    dependsOn: PrepareBuild
    variables:
      download_build_id: $[ dependencies.PrepareBuild.outputs['LatestArtifactsBuild.id'] ]
    steps:
    - checkout: none
    - template: ../steps/download-artifacts-from-pipeline-build.yml
      parameters:
        buildId: $(download_build_id)

Download same pipeline artifact latestFromBranch over multiple jobs

2 Answers2

Linked