How to get the IP Address for Azure DevOps Hosted Agents to add to the white list

Question

Is there a way to the IP address range for the hosted machine running?

This is related to the Release Pipeline -> Hosted agent.

Issue: Getting access denied on connection, as the connection is getting refused via Firewall. Need to whitelist the IP address range for this request coming from release pipeline on DevOps.

score 28 · Answer 1 · answered Jan 09 '19 at 01:50

28

I have a step in a release that gets the Hosted Agent IP address in powershell with:

Invoke-RestMethod http://ipinfo.io/json | Select -exp ip

Hope that helps.

answered Jan 09 '19 at 01:50

David Niwczyk

281
2
4

5

For an Ubuntu based VM this is the equivalent command for e.g. a step as a bash script: `curl -s http://ipinfo.io/json | jq '.ip'`, then you can compare the IP address with the CIDR ranges in the XML file at https://www.microsoft.com/en-nz/download/details.aspx?id=41653 (be aware it changes weekly, as stated in the other answer). Does anyone know if there is a config/setting page on AzureDevops to know this information without running commands? – TPPZ Jan 29 '19 at 11:04
@TPPZ You, me and everybody else is suffering from the same problem. There used to be an API-call (beta/labs) to get agent information, but it has been dropped. Eye-balling the API-docs don't reveal anything useful. – Jari Turkia Mar 04 '19 at 11:41
@JariTurkia if it can help with this pain: whatever thing is on Azure that needs to be accessed by the hosted VM on AzureDevOps e.g. a database can have a firewall rule that allows `0.0.0.0` (cf. https://docs.microsoft.com/en-us/rest/api/sql/firewallrules/createorupdate#request-body). This is some sort of special wildcard in the Azure land to avoid updating IPs due to that XML list I was mentioning. In my scenario my Azure DevOps pipeline with the hosted VM was supposed to create/access a SQL Server instance on Azure. When adding the firewall rule in the SQL Server definition, then all good – TPPZ Mar 04 '19 at 15:21
1

This method looks ok but I cannot get it to work - I'm getting the IP, adding it to NSG but still receiving 403 from Blob Storage Service... – Karpik Oct 03 '19 at 13:26

score 9 · Answer 2 · answered Jan 01 '22 at 09:28

For windows build agents, its safer to use the powershell route:

steps:
- task: AzurePowerShell@5
  displayName: 'Add buildserver public ip'
  inputs:
    azureSubscription: test
    ScriptType: InlineScript
    Inline: |
     $ip = (Invoke-WebRequest -uri "http://ifconfig.me/ip").Content
     New-AzSqlServerFirewallRule -ResourceGroupName "group"  -ServerName "database-server-name" -FirewallRuleName "azuredevops" -StartIpAddress $ip -EndIpAddress $ip
    azurePowerShellVersion: LatestVersion

Sameed · Answer 3 · 2021-10-27T21:41:37.500

The pool gets an update every week from Microsoft. Therefore, it is not possible to maintain it.

For this problem, I had created a new Bash task. The task obtains the current IP from the agent, and whitelists it in the AWS security group. and before finishing the pipeline, I had revoked the said IP.

Current_IP=$(curl ipinfo.io/ip)
echo $Current_IP

# Authorize access
aws ec2 authorize-security-group-ingress \
    --group-id sg-xxxxxxxx \
    --protocol tcp \
    --port 9000 \
    --cidr $Current_IP/32

# Revoke access
aws ec2 revoke-security-group-ingress \
    --group-id sg-xxxxxxx \
    --protocol tcp \
    --port 9000 \
    --cidr $Current_IP/32

And for aws-cli commands to be work, we would also need a restricted aws policy that will allow the above command to be run with the limited access.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:RevokeSecurityGroupIngress",
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:ModifySecurityGroupRules"
            ],
            "Resource": "arn:aws:ec2:us-east-2:xxxxxx:security-group/sg-xxxxx"
        }
    ] 
}

I have a self-managed service that is hosted on AWS. Once my CI pipeline is completed, I needed to send the report to that desired service which is only accessible to the whitelisted IP's. — Sameed, Oct 28 '21 at 08:45
Partially. The question ask's to get the Azure agent IP which need's to be whitelisted somewhere. I not only answered to get an agent IP, but also described the process of getting the IP and whitelisting it to the AWS, which was my use case. — Sameed, Oct 29 '21 at 11:22

score 4 · Answer 4 · answered Nov 22 '18 at 06:10

4

Use a script step in the pipeline to get the current external ip and whitelist it. after pipeline finishes use another script step to clean up.

Thats the only way (for hosted agent), unfortunately.

answered Nov 22 '18 at 06:10

4c74356b41

69,186
6
100
141

2

There is an article article around the IP ranges for VSTS/AzureDevOps used by the Microsoft Azure Datacenters (https://www.microsoft.com/en-nz/download/details.aspx?id=41653) which change every week. There is a PublicIP xml file for each region which we need to whitelist during any deployment – Gagan Jeet Singh Nov 26 '18 at 22:54
3

this is a real overkill – 4c74356b41 Nov 27 '18 at 04:42
Thanks for the update and help. The above answer might not work as the customer has to know the IP address up-front in order to **White list** this. Getting the IP address from the script and then wait for the customer to add it to the White List will not work. So white listing the IP range worked for us. – Gagan Jeet Singh Nov 28 '18 at 21:37
2

might as well whitelist 0.0.0.0\0 – 4c74356b41 Nov 29 '18 at 05:36
"Getting the IP address from the script and then wait for the customer to add it to the White List will not work." Just add it with a script inside the pipeline. – Datautomate Aug 25 '22 at 12:19

score 3 · Answer 5 · answered May 29 '19 at 12:53

3

Check out this add on for Azure DevOps (https://marketplace.visualstudio.com/items?itemName=MartijnQuekel.AzureAppServiceIPRestrictions). It allows you to alter your App Service IP restrictions during the build pipeline.

answered May 29 '19 at 12:53

Nigel Gossage

95
8

This still works but uses a deprecated API which seems to be returning more warning messages with every passing month. Has anyone found an up-to-date version or a similar alternative? – keithl8041 Feb 25 '21 at 18:12

score 2 · Accepted Answer · answered Nov 26 '18 at 22:56

2

We need to white list the IP address used by the Azure Datacenters in the list mentioned below: https://www.microsoft.com/en-nz/download/details.aspx?id=41653

Note: This list gets updated every week, so please be mindful of this during the deployment planning

answered Nov 26 '18 at 22:56

Gagan Jeet Singh

241
1
2
7

If you need to white list these IPs in the firewall rules for other Azure services, then you could just use the work around of start/stop IP range `0.0.0.0` as explained here: https://learn.microsoft.com/en-us/rest/api/sql/firewallrules/createorupdate#request-body this way you don't need to update the IP list according to the XML downloaded weekly from https://www.microsoft.com/en-nz/download/details.aspx?id=41653 – TPPZ Jan 29 '19 at 11:39
Did this work for you? I am facing similar issue. I need to deploy sql changes to client's database using pipelines. Its not a good idea to ask client to update the IP range every week. :( – Priya Apr 09 '20 at 09:09
I don't think this answer is valid anymore. The link is now hitting a "[Deprecated]" download. – Matt Oct 15 '20 at 14:51

srsn · Answer 7 · 2021-11-29T12:33:49.833

I agree with what @4c74356b41 has mentioned.

I have been using the below solution to add the Azure DevOps IP addresses to Authorized IP ranges for Azure Kubernetes Service. You can modify this solution to use it for any of the Azure Services.

You need to add two "Azure Cli" tasks - one to add Azure DevOps Agent IP address and another to remove the Azure DevOps Agent IP Address.

Add the first task before the Kubernetes tasks:

I added a new task in Azure DevOps pipeline with "Azure Cli" and added the below commands as inline script:

echo "Get Azure DevOps IP address"

azdoip=`curl -s icanhazip.com`

echo "Azure DevOps IP Address: $azdoip"

echo "Set Azure Subscription to MYSUBSCRIPTION"

az account set  --subscription "MYSUBSCRIPTION"

echo "Get credentials for AKS Cluster Admin"

az aks get-credentials --resource-group MYAKSRG --name MYAKSCLUSTER --admin --file  ~/.kube/config

Echo "Get existing authorized ip ranges"

authorizedips=`az aks show  --resource-group MYAKSRG  --name MYAKSCLUSTER  --query apiServerAccessProfile |jq -r '.authorizedIpRanges | join(",")'`

echo "Update Azure DevOps IP Address in AKS Cluster Authorized IP Ranges"

az aks update  --resource-group MYAKSRG  --name MYAKSCLUSTER  --api-server-authorized-ip-ranges $authorizedips,$azdoip

Once all the kubernetes tasks has been finished, add another "Azure Cli" task at the end to remove Azure DevOps IP.

echo "Set Azure Subscription to MYSUBSCRIPTION"

az account set  --subscription "MYSUBSCRIPTION"

echo "Get credentials for AKS Cluster Admin"

az aks get-credentials --resource-group MYAKSRG --name MYAKSCLUSTER --admin --file  ~/.kube/config

echo "Get New Authorized IP Ranges"

newauthorizedips=`az aks show  --resource-group MYAKSRG   --name MYAKSCLUSTER  --query apiServerAccessProfile |jq -r '.authorizedIpRanges | join(",")'`

echo "Remove AzDo IP and store it as a variable" #Removes last element from the array of IPs

authorizedips=`echo $newauthorizedips | awk 'BEGIN{FS=OFS=","}NF--'`

echo "Update AKS by restoring Authorized IPs"

az aks update  --resource-group MYAKSRG   --name MYAKSCLUSTER --api-server-authorized-ip-ranges $authorizedips

One problem with this approach as it relates to AKS is that, unless you subsequently remove the new IP at the end of each build eventually you will hit the 200 IP address limit of this feature: https://learn.microsoft.com/en-us/azure/aks/api-server-authorized-ip-ranges. Another challenge you'll need to deal with is that these rules can take up to 2min to propagate. It may be better in AKS to run a self-hosted build agent on your cluster to keep it secure and avoid these problems. — Adam Hems, Jun 01 '21 at 15:17
@AdamHems I have updated my solution to remove the Azure DevOps IP at the end of the deployment pipeline. This is a simple solution (but no elegant) to overcome the problem. I agree the better solution is to use self hosted agents — srsn, Nov 29 '21 at 12:28

score 1 · Answer 8 · answered Apr 08 '22 at 16:39

The problem, as you may have encountered, is the IP not only changes each run, but also changes each week.

This was a problem for us because we needed to add the whitelist of these IP's to our security group in Terraform so the pipeline could SSH into our container and deploy. We did look at using something like the above which was getting the IP from the machine by calling a 3rd party and storing as a var, but this was tricky since we needed to apply this in Terraform

More problems include:

URL for downloading the IP address from Microsoft changes each time. So its hard to point to a static URL
Looping through the IPs and making the data readable for terraform required a bit of horrible replacing and splitting
Also worth noting, data from external data is not really designed for being used like this, so proceed with caution.

If you or anyone is using Terraform, this is what we came up with:

terraform/.../security.tf

data "external" "azure_devops_ip_ranges" {
  program = ["bash", "${path.root}/../azure_devops_get_ip_ranges.sh"]
}

resource "...._security_group" "azuredevops" {
  name        = "azuredevops"
  description = "access from azuredevops"
}

resource "...._security_group_rules" "azuredevops" {
  security_group_id = ...._security_group.azuredevops.id

  ## had to increase the timeout here because the amount of IPs from Azure Devops was in the hundreds and it took time
  timeouts {
    create    = "15m"
    update    = "15m"
    delete    = "5m"
  }

  ## So yep, this looks a little ugly. We get the data from the external call with data.external.azure_devops_ip_ranges.result.ips. Then replace quotes with nothing so its just the IPs. 
  ## Then split over a new line
  ingress {
    protocol  = "TCP"
    ports     = ["22"]
    cidr_list = split("\n", replace(data.external.azure_devops_ip_ranges.result.ips, "\"", ""))
  }

}

terraform/azure_devops_get_ip_ranges.sh

#!/bin/bash

## Scrape the download page for the download link
DOWNLOAD_LINK=$(curl -sS https://www.microsoft.com/en-us/download/confirmation.aspx\?id\=56519 | egrep -o 'https://download.*?\.json' | uniq)

## curl the download link and use jq to get the IPs we need. For pipelines you need AzureCloud and our region was uksouth
IPS=$(curl ${DOWNLOAD_LINK} | jq '.values[] | select(.name=="AzureCloud.uksouth") | .properties.addressPrefixes'[])

## because using external data in terraform expects JSON, output JSON
jq -n \
    --arg ips "$IPS" \
    '{"ips":$ips}'

Hope this can help someone

for Azure users, that can also work like this: `IPS=$(az network list-service-tags --location francecentral --query "values[?name == 'AzureCloud.centralfrance'].properties.addressPrefixes[]")` — Loul G., May 11 '22 at 14:10

score 0 · Answer 9 · answered Jun 26 '19 at 10:10

In case you came looking here becasue you're getting this error when trying to use Azure DevOps to MSBuild and deploy to an Azure SQL server and going slightly mad because nothing seems to be on the internet and people are all taking about using power shell scripts to find out the IP address of the server and white listing etc.., then you may be better of using a task called "Azure SQL Database deployment" in your yml file as well as the MSBuild like so:

- task: MSBuild@1
  displayName: Build the database project
  inputs:
    solution: '**/projectname.sqlproj'
    msbuildArguments: '/t:Restore /t:Build '



- task: SqlAzureDacpacDeployment@1
  inputs:
    azureSubscription: ''
    AuthenticationType: 'server'
    ServerName: '.database.windows.net'
    DatabaseName: ''
    SqlUsername: ''
    SqlPassword: ''
    deployType: 'DacpacTask'
    DeploymentAction: 'Publish'
    DacpacFile: '**/projectname.dacpac'
    IpDetectionMethod: 'AutoDetect'

the IpDetectionMethod of auto worked for me put it does allow you to easily put in your own values (although i haven't tried that)

score 0 · Answer 10 · answered Jul 31 '22 at 09:22

0

You can use Azure DevOps whiting list

See @Leo Liu-MSFT answer: Whitelisting Azure Devops IPs for Service Hooks

answered Jul 31 '22 at 09:22

Shoshana Tzi

99
4
10

How to get the IP Address for Azure DevOps Hosted Agents to add to the white list

10 Answers10

Linked