Azure Pipeline YAML templates - Is there any way to check and confirm if a pipeline covers all the required jobs while using a template?

Question

We are using yaml pipelines in Azure Devops along with templates. The requirement is to identify if all the pipeline that uses the template are running a required set of steps or not? Is there any way to confirm this, other than manual monitoring.

It would have been useful if conditional checks can be added so that we can check if a specific task is present or not.

To explain as an example, let's say that a template has 4 tasks for running 4 different types of tests. Multiple pipelines are created using this template. They can opt to run these tests by turning it ON (set Yes/No value in input parameter). We need to check and verify if all the pipelines are running all these 4 tests.

Answer 1

There is no easy way to do this, as we don't have an option to get details about used teplates via REST API, or any other way. What you can do is try to template this a bit. So first you need to create a template like:

paramaters:
- name: repositoryName
  type: string
  default: 'self'    
- name: pipelinePath
  type: string
  default: ''  # if your pipeline file has always the same name you can put it here and do not set calling pipeline   

jobs:
- job: '${{ parameters.repositoryName }}' # please easnure that you repository name is a valid job name
  dependsOn: []
  steps:
  - checkout: ${{ parameters.repositoryName }}
  - pwsh: |
      Here you should use yq tool to extract steps which uses your template
      Then manipulate it to be sure that it checks you conditions
      And then send some alerts to slack or wahtever you want

Then you can call it from the pipeline like:

parameters:
 - name: repositories
  type: object
  default:
  - RepoA
  - RepoB
  - list all your repos here

resources:
  repositories:
  - repository: RepoA
    type: git
    name: RepoA
  - repository: RepoB
    type: git
    name: RepoB
  - list the same repos here to be sure you have permission to use them in the pipeline

jobs:
# I assumed that `pipelinePath` is always the same and set as default an thus we can use each expression. If not you will have to call it one by one
- ${{ each repository in parameters.repositoryName }}: 
  - template: template.yml
    paramaters:
      repositoryName: ${{ repository  }} 

Here is the potential solution. I can't provide full but I think that idea is clear. I suggested to use yq tool but you can use whatever you want what help you to verify that templates is used as expected. Also, you can setup cron job to have it run on schedule.