2

We have a self-hosted build agent on an on-prem server.

We typically have a large codebase, and in the past followed this mechanism with TFS2013 build agents:

  • Daily check-ins were built to c:\work\tfs\ (taking about 5 minutes)
  • Each night a batch file would run that did the same build to those folders, using the same sources (they were already 'latest' from the CI build), and build the installers. Copy files to a network location, and send an email to the team detailing the build success/failures. (Taking about 40 minutes)

The key thing there is that for the nightly build there would be no need to get the latest sources, and the disk space required wouldn't grow much. Just by the installer sizes.

To replicate this with Azure Devops, I created two pipelines. One pipeline that did the CI using MSBuild tasks in the classic editor- works great Another pipeline in the classic editor that runs our existing powershell script, scheduled at 9pm - works great

However, even though my agent doesn't support parallel builds what's happening is that: The CI pipeline's folder is c:\work\1\ The Nightly build folder is c:\work\2\

This doubles the amount of disk space we need (10gb to 20gb) They are the same code files, just built differently.

I have struggled to find a way to say to the agent "please use the same sources folder for all pipelines"

What setting is this, as we have to pay our service provider for extra GB storage otherwise.

Or do I need to change my classic pipelines into Yaml and somehow conditionally branch the build so it knows it's being scheduled and do something different? Or maybe, stop using a Pipeline for the scheduled build, and use task scheduler in Windows as before?

(I did try looking for the same question - I'm sure I can't be the only one).

GilesDMiddleton
  • 2,279
  • 22
  • 31

2 Answers2

2

The number '1' '2'...'6' of work folder c:\work\1\, c:\work\2\... c:\work\6\ in your build agent which stands for a particular pipeline.

Agent.BuildDirectory

The local path on the agent where all folders for a given build pipeline are created. This variable has the same value as Pipeline.Workspace. For example: /home/vsts/work/1

If you have two pipelines, there will also be two corresponding work folders. It's an except behavior. We could not configure pipelines to share the same build folde. This is by designed.

If you need to use less disk space to save cost, afraid stop using a Pipeline for the scheduled build, and use task scheduler in Windows as before is a better way.

PatrickLu-MSFT
  • 49,478
  • 5
  • 35
  • 62
  • Thank you for your input but that doesn't really move us on into the future. Being able to force the pipeline to share the same build directory with another pipeline on the same agent when the source control directive is identical - would have been a wonderful design feature to speed up builds. Shame it's been designed out. I'll be trying to make my PowerShell script change to the other pipeline's directory. – GilesDMiddleton Jun 17 '20 at 14:31
  • @GilesDMiddleton Thanks for your kindly update. We were trying to make the folder structure more brief and clearly. Each pipeline corresponding to a different folder will be easy to manage download build source, build logs, generated build files. All the changes are encapsulated in each pipeline with no interference to or from all the others. But if you found a workaround solution, feel free to share it here which may help others in the community. – PatrickLu-MSFT Jun 19 '20 at 08:54
1

There is "workingDirectory" directive available for running scripts in pipeline. This link has details of this - https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/command-line?view=azure-devops&tabs=yaml

Kamal
  • 51
  • 4
  • Interesting - even If this doesn’t work, You’ve made me rethink what I’m doing as well. Just because my power shell script runs with relative paths, there’s no reason why I can’t hard code it to detect it’s being run in 2/s and force it to flip to 1/s - I’ll give it a go, thanks. – GilesDMiddleton Jun 11 '20 at 11:45