13

I'm using TFS 2012 Update 1 on Windows Server 2008 R2. We have the TFS build running under the account TFSUser. The other day we ran out of space on our build machine. Upon investigation I found that the folder C:\Users\TFSUser\AppData\Local\Temp had over 50GB of files in it, some dating back as far as October of 2012. None of the files are overly large, but they don't ever seem to get cleaned up.

Investigating today shows that folder gets written to a lot during an automated build. Why aren't these files getting cleaned up and what can I do to ensure my build machine doesn't run out of space again due to this issue?

Update 2013-03-13
I created a small powershell script that runs nightly to delete the temp directory contents. Here's the powershell script:

Stop-Service TFSBuildServiceHost.2012
Remove-Item Drive:\Path\To\TFSUser\AppData\Local\Temp\* -recurse -exclude Build*
Start-Service TFSBuildServiceHost.2012

I have the task run with elevated privileges nightly as the TFSUser account. The elevated privileges are needed because we need to start and stop services.

Corith Malin
  • 1,505
  • 10
  • 18

1 Answers1

10

This is a TFS2010 issue that I guess they haven't addressed in 2012 yet. The build system creates temp files and never cleans them up, leading eventually to failures because the disk is full or because they use a limited number of digits for the unique IDs and run out of new ids that can be used (IIRC, it falls over at 65536 files, which we usually hit long before we run of disk space)

The solution is to delete the temp files periodically. I just do this manually about 2-4 times a year. The safest approach is probably to disable your build agents and then clean out the temp folder, then re-enable the agents (and it may not be a bad idea to reboot the server at the same time if it's been running for months).

An alternative may be to set up a small scheduled task daily/weekly that cleans away any temp files older than a few days old, and run it in the middle of the night when you know nothing else (like overnight builds) will be running.

Or if your server is rebooted several times a year anyway, a start-up script that cleans out the temp folder before the build agents start up may be a clean & tidy solution.

It all depends how quickly your disk is filling up - you'll probably find that it's many months of accumulated cruft rather than something that hits you after a short time, in which case you don't need a very aggressive approach to keep things running smoothly.

Jason Williams
  • 56,972
  • 11
  • 108
  • 137
  • Thanks Jason. I'll look into a small script that gets run as I'm quite forgetful (hence why I program). :-) – Corith Malin Mar 13 '13 at 13:20
  • 1
    Turns out I'm quite forgetful too :-) ... I just checked our server this morning and I'd set up a scheduled task that runs every Saturday night at 11:30pm, and runs a 2-line batch script: `del /F /S /Q "C:\Documents and Settings\TFSSERVICE\Local Settings\Temp\*.*"` `del /F /S /Q "C:\WINDOWS\Temp\*.*"` – Jason Williams Mar 13 '13 at 15:07
  • We hit this issue when our vdproj Setup projects started failing on the TFS 2010 build server. Devenv attempts to create temp files when building the MSI. Only found this snag out by launching VS and creating a new MSI project/vdproj (on the build server), whilst at the same time using ProcMon and subsequently noticed a huge raft of *NAME COLLISION* results on CreateFile operations in the build service's AppData\Local\Temp folder. Clearing this out now... – Wim Jun 11 '13 at 11:18
  • If you only allow one build to run at a time on the same machine then you can set up a build which clears out temp – paulm Mar 10 '14 at 16:17
  • 1
    Why on earth are we having to do stuff like this in 2014? Before I knew TFS was doing this I had >118K MSI log files alone at over 279GB. – Adam Caviness Apr 02 '14 at 19:32