3

I have the following configuration: Azure DevOps Server Version Dev18.M170.8

trigger: none # No CI build
pr: none # Not for pull requests

schedules:
- cron: "* * * * *"
  displayName: Every minute build
  branches:
    include:
    - master
  always: true
  
jobs:
- job: RunMyCustomTask
  pool: 'DedicatedPoolWith1Agent'
  condition: always()
  steps:
  - task: mycustomtask@0'

The pipeline is created for a separate repo and has a dedicated agent pool. It should run, normally, every minute, based on the configuration. This is critical for me. I can change to every 2 minutes or 5 minutes, but it does not matter, it must be scheduled precisely at the right intervals.

Now, I have the issue that the pipeline is scheduled every minute, and then, for no whatsoever reason, it is not scheduled anymore. I have checked this during a day, and it happened that it wasn't scheduled for 6 minutes the most. I do not understand why this is happening. Any ideas or suggestions? I need schedules to be guaranteed. Trigger and pr: none makes no difference, with or without them. Image example: scheduled pipeline not executing 6 minutes

Agent execution logs

XMight
  • 1,991
  • 2
  • 20
  • 36
  • Is there a chance that you don't have enough agents? Can you go to agent details and check `Wait time`? – Krzysztof Madej Jul 06 '21 at 14:26
  • @KrzysztofMadej as I mentioned in the question, I have a dedicated pool and a dedicated single agent just for this pipeline. No other pipelines use this agent or pool. In theory, the Agent should get a new scheduled pipeline run every minute and executed it, with a duration of maximum 1 minute, and normally under 1 minute start to end. Anyway, this does not explain why Azure DevOps Server does not schedule the pipeline for 6 minutes. If there were agent issues, I would have had piling scheduled runs, not gaps. – XMight Jul 06 '21 at 14:30
  • What do you see when you click on [scheduled runs button](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-runs-view)? – Krzysztof Madej Jul 06 '21 at 14:31
  • @KrzysztofMadej I added a screenshot for the Agent Jobs, with highlight of the moment with issue. – XMight Jul 06 '21 at 14:35
  • Is it the same when you use MS Hosted agents? Is ti possible for you to check? – Krzysztof Madej Jul 06 '21 at 14:37
  • Regarding the scheduled runs button, it shows next 10 schedules correctly. But they are not executed when they are scheduled (Sometimes obviously). Regarding MS Hosted, yes, I will do that. The problem is that I cannot use MS Hosted Agents always, because of OnPremises software. – XMight Jul 06 '21 at 14:37

1 Answers1

0

I'm not sure that Azure DevOps pipelines is the appropriate place to be running frequent, short-lived processes on a short timer.

The documentation even states that there is a limit of about 1000 jobs per week: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#limits-on-the-number-of-scheduled-runs

There are certain limits on how often you can schedule a pipeline to run. These limits have been put in place to prevent misuse of Azure Pipelines resources - particularly the Microsoft-hosted agents. This limit is around 1000 runs per pipeline per week.

Based on your statement, it must be scheduled precisely at the right intervals, I'd reconsider whether pipelines are the right tool for this job. There is no real-time guarantee with scheduled pipelines.

Daniel Mann
  • 57,011
  • 13
  • 100
  • 120
  • 2
    I understand about Microsoft Hosted, but I'm running in an On-Premises Environment with my own resources. How your answer relates to my problem? – XMight Jul 06 '21 at 14:54
  • 2
    Additionally, I stated clearly that I can change the frequency, but I need the times/intervals to be guaranteed. I don't necessarily need 1 minute cron scheduled pipeline runs. – XMight Jul 06 '21 at 15:05
  • 1
    @XMight The limit is regardless of hosted or private agent. And like I said, **there is no real-time guarantee**. It will do its best to run the pipeline at the scheduled time, but it may not run immediately at that time for a variety of reasons (service unavailability, agent unavailability, etc). – Daniel Mann Jul 06 '21 at 15:07
  • It makes sense what @DanielMann is saying. You can create maybe another question and if you provide more details what actually do you need to run on scheduled manner maybe we will find a proper approach for you. – Krzysztof Madej Jul 06 '21 at 15:16