Cloning all remote repositories from TFS

Question

Is there a way to clone all the master branches from all the projects uploaded under 1 account.

I have a requirement to backup all the code in master branch every week. Is there a way to accomplish this using git, Powershell or any other way?

Note that I required to do this task in a Windows environment.

score 10 · Accepted Answer · edited Dec 08 '22 at 20:03

You can do it with PowerShell and TFS Rest API.

First of all, get the projects with Projects - List API, then get to each project the repositories with Repositories - List API and clone only the master.

For example, a simple PowerShell script that do it:

$collectionUrl = "http://tfs-server:8080/tfs/collection-name"

$projectsUrl = "$collectionUrl/_apis/projects?api-version=4.0"
$projects = Invoke-RestMethod -Uri $projectsUrl -Method Get -UseDefaultCredentials -ContentType application/json

$projects.value.ForEach({

    $reposUrl = "$collectionUrl/$($_.name)/_apis/git/repositories?api-version=4.0"
    $repos = Invoke-RestMethod -Uri $reposUrl -Method Get -UseDefaultCredentials -ContentType application/json
    $repos.value.ForEach({
    git clone $_.remoteUrl --branch master --single-branch

    })
})

If you're using an older version of TFS, I can confirm this also works for api-version=1.0 — Ben Pittoors, Feb 16 '23 at 08:51

score 1 · Answer 2 · answered Aug 29 '19 at 04:44

I did not see that feature directly in TFS, but if the VSTS API is also available for your on-premise TFS instance, you can:

use the API to get all projects, and then all repositories for a given account
use that list to loop and pull each master branch in a local clone of those repositories, that you can then archive.

score 1 · Answer 3 · answered Apr 26 '21 at 20:13

I've created a github gist that accomplishes this in Azure DevOps. You can find it here. I used this to set up my new machine after refreshing windows or something equivalent.

Code as it stood on original commit:

#Ensure you create a PAT, following the instructions here: https://dev.to/omiossec/getting-started-with-azure-devops-api-with-powershell-59nn
#Additional Credit: https://blog.rsuter.com/script-to-clone-all-git-repositories-from-your-vsts-collection/
#I suggest executing from C:/Projects. This script will create a folder for each Team Project/Client with repos within each.
#Finally note that git clone operations count as "Errors" in powershell, and appear red. It's more work than is worth it to change it.

param(
    [string] $email = $(Throw "--Email is required."), #required parameter
    [string] $pat = $(Throw "--PAT is Required"), #required parameter
    [string] $url = $(Throw "--Collection URL is required.") #required parameter pointing to the default collection, URL is generally https://{tenant}.visualstudio.com/defaultcollection
)
$originalDirectory = Get-Location

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $email,$pat)))
$headers = @{
    "Authorization" = ("Basic {0}" -f $base64AuthInfo)
    "Accept" = "application/json"
}

Add-Type -AssemblyName System.Web
$gitcred = ("{0}:{1}" -f  [System.Web.HttpUtility]::UrlEncode($username),$password)

#Write-Host "Retrieving Repositories...`n"
#$resp = Invoke-WebRequest -Headers $headers -Uri ("{0}/_apis/git/repositories?api-version=1.0" -f $url)
#$repoJson = convertFrom-JSON $resp.Content
#Write-Host $repoJson

Write-Host "Getting Projects..."
$projectsUrl = "$collection/_apis/projects?api-version=4.0"
$projectResponse = Invoke-Webrequest -Headers $headers -Uri ("{0}/_apis/projects?api-version=4.0" -f $url)
$projects = ConvertFrom-Json $projectResponse

$projects.value.ForEach({
    $folderToCreate = Join-Path -Path $originalDirectory -ChildPath $_.name

    if (!(Test-Path $folderToCreate -PathType Container)) {
        Write-Host "Creating folder for Project $($_.name)"
        New-Item -ItemType Directory -Force -Path $folderToCreate
    } else {
        Write-Host "Skipping folder creation for project $($_.name), as it already exists"
    }
    Set-Location $folderToCreate

    $reposUrl = "$url/$($_.name)/_apis/git/repositories?api-version=4.0"
    $reposResponse = Invoke-Webrequest -Headers $headers -Uri $reposUrl
    $repos = ConvertFrom-Json $reposResponse    

    $repos.value.ForEach({
        $name = $_.name
        Write-Host "Cloning $name Repos"

        try {            
            $credUrl = $_.remoteUrl -replace "://", ("://{0}@" -f $gitcred)
            git clone $credUrl --branch master --single-branch    
            #git clone $_.remoteUrl --branch master --single-branch #this will automatically create/use GitForWindows token after a login prompt if you have issues with the upper 2 lines
        }
        catch {
            Write-Host $PSItem.Exception.Message -ForegroundColor RED
            Write-Host "Error at URL $_.remoteUrl"
            Set-Location $originalDirectory
        }        
    })
    Write-Host "Cleaning URL Space encoding for repo folders..."
    Get-ChildItem $folderToCreate | 
        Where {$_.Name -Match '%20'} | 
            Rename-Item -NewName {$_.name -replace '%20',' ' } #Rename-Item  { $_.Name -replace "%20"," " }
    Set-Location $originalDirectory
})

Cloning all remote repositories from TFS

3 Answers3