5

As part of my build process in VSTS I want to delete all files and folders (except af few) from my azure site before a new deploy. My guess is, that using a Azure Powershell script would be a good idea and I would prefer making an inline script.

I am using Azure Resource Manager as connection type, I have selected my subscription and script type (Inline Script) but then I am lost, how do i select my app service and, for a start, list my files?

Trying just, this for a test, only gives my files in my VSTS environment

Get-ChildItem -Path $(build.sourcesDirectory)
keysersoze
  • 777
  • 3
  • 12
  • 27
  • Is this an azure web app from where you want to clear certain files/folders? – degant May 05 '17 at 09:44
  • vsts builds take the latest version from the source control/git and publishes to azure resources which in this case is a web app. are you looking for upgrading your web app ? it is same as publishing your changed web app again on the same azure web app. – Aravind May 05 '17 at 09:55
  • @degant Correct – keysersoze May 06 '17 at 11:24
  • @Aravind No, I want to delete everything (almost) on my site before a publish, it is a test environment, so downtime is not an issue. Unfortunately I cannot use the "delete everything not in this package" option at deploy because there are 1 folder and 2-3 config-files that are not in my package and which should remain. The reason a delete is necessary is to avoid old files. – keysersoze May 06 '17 at 11:25
  • A new build and publish would do that for you. No need to delete anything. – Aravind May 06 '17 at 16:14
  • @Aravind No? If my project have file something.txt and I deploy it and then later I delete it from my project and do new a deploy something.txt is still online. Only if I check "Remove additional files at destination" something.txt is removed, but then it also removes files and folders I do need and which cannot be part of my project. – keysersoze May 07 '17 at 17:29
  • @keysersoze you can just do a fresh deployment. whatever you include in your build will get published. If you do not need something you do not include that in your build. – Aravind May 08 '17 at 06:22
  • @Aravind Sorry, then I am not sure what you mean with a fresh deployment compared to my current running deployment. In short, when I push to VSTS it triggers a new build (Visual Studio Build) followed by a new deploy (Azure App Service Deploy) - this puts, as expected, all necessary files to Azure App Service; Existing files are overwritten but files no longer in my project remains. How differs your idea from what I am doing? – keysersoze May 08 '17 at 08:03
  • @keysersoze if you add some files via kudu, ftp or some manual way other than the VSTS build process those will not be deleted. it is not a good practice to add files outside of a source control or CI process. – Aravind May 08 '17 at 09:22
  • @Aravind It is good practice if the files have nothing to do with your project - in this case it is a hosted CMS so I cannot completely myself decide what is on the server, I just have instructions not to delete certain license and analytics files and these files differs from one environment to another so even if i added them to my project I would make a mess. – keysersoze May 08 '17 at 13:34
  • My approach would be to clear all files from the server before a deployment, deploy the application files and then deploy the licence files. The licence files don't need to live in the project but can be uploaded from another source. – Padraic May 09 '17 at 10:45
  • What's the result of deleting folder with Kudu api? – starian chen-MSFT May 12 '17 at 07:36
  • @starain-MSFT No perfect solution yet so still looking a bit into it - but for now your answer does the trick. – keysersoze May 28 '17 at 09:04

3 Answers3

6

First, it’s better to include the files to the project that the web app needs, then just check Remove additional files at destination option (Check Publish using Web Deploy option first) to remove additional files.

Secondly, you can remove the files through Kudu API.

DELETE /api/vfs/{path}    (Delete the file at path)

More information, you can refer to: Interacting with Azure Web Apps Virtual File System using PowerShell and the Kudu API

Update (Add Kudu sample):

  1. Add Azure PowerShell step/task
  2. Sepcify arguments, for example: -resourceGroupName XXX -webAppName XXX -kuduPath Global.asax

Script:

param(
    [string]$resourceGroupName,
    [string]$webAppName,
    [string]$slotName="", 
    [string]$kuduPath
)
function Get-AzureRmWebAppPublishingCredentials($resourceGroupName, $webAppName, $slotName = $null){
    if ([string]::IsNullOrWhiteSpace($slotName)){
        $resourceType = "Microsoft.Web/sites/config"
        $resourceName = "$webAppName/publishingcredentials"
    }
    else{
        $resourceType = "Microsoft.Web/sites/slots/config"
        $resourceName = "$webAppName/$slotName/publishingcredentials"
    }
    $publishingCredentials = Invoke-AzureRmResourceAction -ResourceGroupName $resourceGroupName -ResourceType $resourceType -ResourceName $resourceName -Action list -ApiVersion 2015-08-01 -Force
    Write-Host $publishingCredentials   
    return $publishingCredentials
}
function Get-KuduApiAuthorisationHeaderValue($resourceGroupName, $webAppName, $slotName = $null){
    $publishingCredentials = Get-AzureRmWebAppPublishingCredentials $resourceGroupName $webAppName $slotName
    Write-Host $publishingCredentials.Properties.PublishingUserName
    Write-Host $publishingCredentials.Properties.PublishingPassword
    return ("Basic {0}" -f [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $publishingCredentials.Properties.PublishingUserName, $publishingCredentials.Properties.PublishingPassword))))
}
function Delete-FileToWebApp($resourceGroupName, $webAppName, $slotName = "", $kuduPath){

    $kuduApiAuthorisationToken = Get-KuduApiAuthorisationHeaderValue $resourceGroupName $webAppName $slotName
    if ($slotName -eq ""){
        $kuduApiUrl = "https://$webAppName.scm.azurewebsites.net/api/vfs/site/wwwroot/$kuduPath"
    }
    else{
        $kuduApiUrl = "https://$webAppName`-$slotName.scm.azurewebsites.net/api/vfs/site/wwwroot/$kuduPath"
    }

    Write-Output $kuduApiUrl
    Write-Output $kuduApiAuthorisationToken
    Invoke-RestMethod -Uri $kuduApiUrl `
                        -Headers @{"Authorization"=$kuduApiAuthorisationToken;"If-Match"="*"} `
                        -Method DELETE
}

Delete-FileToWebApp $resourceGroupName $webAppName $slotName $kuduPath
starian chen-MSFT
  • 33,174
  • 2
  • 29
  • 53
  • The reason I cannot include the files in the project is that the files do not belong in the project - this includes some license files (that differs from one environment to another) but also files and folders from some analytics stuff (New Relic). Otherwise of course I would include it in my project. I will take a look at Kudu. – keysersoze May 08 '17 at 08:07
  • @keysersoze You can just include the additional files in the package file, you can use this extension to zip file https://marketplace.visualstudio.com/items?itemName=petergroenewegen.PeterGroenewegen-Xpirit-Vsts-Build-Zip – starian chen-MSFT May 08 '17 at 08:13
  • If I include the Analytics-stuff in my project and it is later updated i risk overwriting it with my old files? Or do you want me to create a task to download the files from azure and include before I deploy? – keysersoze May 08 '17 at 09:13
  • @keysersoze I mean download/map the files to build agent and include to package file before deploy. Just depend on which easy to do, so you can choose what you like. – starian chen-MSFT May 08 '17 at 09:29
  • Tnx! it looks a bit like the way I was trying to go but I keep getting "The remote server returned an error: (401) Unauthorized.". User and pass matches what I have in my publish settings so that part looks correct but Invoke-RestMethod fails - do I need special Kudu rights or? – keysersoze May 09 '17 at 10:04
  • Ah, got it - its because username contains $, problem solved. But as far as I can see there is no simple way using Kudu to "delete all except". – keysersoze May 09 '17 at 10:55
  • @keysersoze You can delete the folder directly by using DELETE /directory/path/with/slash/. For example: https://tempappstarain.scm.azurewebsites.net/api/vfs/site/wwwroot/bin/?recursive=true (add recurisive=true parameter will delete folder and files recursive) https://github.com/c9/vfs-http-adapter – starian chen-MSFT May 10 '17 at 01:51
  • this is a great script, however it cannot delete folders. ah, with recursive it works :) – vip32 Jul 05 '17 at 13:25
6

Here's a tweaked version of the script which should be included in your project and exported as an artefact as part of your build, I call mine Delete-WebAppFiles.ps1

It expands on the previous answer by also handling virtual applications and having error handling for the case when the files do not exist e.g. on first deployment to a new environment

param(
    [string]$resourceGroupName,
    [string]$webAppName,
    [string]$appPath="wwwroot",
    [string]$slotName="", 
    [string]$kuduPath,
    [bool]$recursive=$false
)
function Get-AzureRmWebAppPublishingCredentials($resourceGroupName, $webAppName, $slotName = $null){
    if ([string]::IsNullOrWhiteSpace($slotName)){
        $resourceType = "Microsoft.Web/sites/config"
        $resourceName = "$webAppName/publishingcredentials"
    }
    else{
        $resourceType = "Microsoft.Web/sites/slots/config"
        $resourceName = "$webAppName/$slotName/publishingcredentials"
    }
    $publishingCredentials = Invoke-AzureRmResourceAction -ResourceGroupName $resourceGroupName -ResourceType $resourceType -ResourceName $resourceName -Action list -ApiVersion 2015-08-01 -Force
    Write-Host $publishingCredentials   
    return $publishingCredentials
}
function Get-KuduApiAuthorisationHeaderValue($resourceGroupName, $webAppName, $slotName = $null){
    $publishingCredentials = Get-AzureRmWebAppPublishingCredentials $resourceGroupName $webAppName $slotName
    Write-Host $publishingCredentials.Properties.PublishingUserName
    Write-Host $publishingCredentials.Properties.PublishingPassword
    return ("Basic {0}" -f [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $publishingCredentials.Properties.PublishingUserName, $publishingCredentials.Properties.PublishingPassword))))
}
function Delete-KuduFile($resourceGroupName, $webAppName, $appPath, $slotName, $kuduPath, $recursive){

    $kuduApiAuthorisationToken = Get-KuduApiAuthorisationHeaderValue $resourceGroupName $webAppName $slotName
    if ($recursive -eq $true) {
        if (-not ($kuduPath.endswith("recursive=true"))) {
           if (-not ($kuduPath.endswith("/"))) {
              $kuduPath += "/"
           }
           $kuduPath += "?recursive=true"
        }
    }
    if ($slotName -eq ""){
        $kuduApiUrl = "https://$webAppName.scm.azurewebsites.net/api/vfs/site/$appPath/$kuduPath"
    }
    else{
        $kuduApiUrl = "https://$webAppName`-$slotName.scm.azurewebsites.net/api/vfs/site/$appPath/$kuduPath"
    }

    Write-Output $kuduApiUrl
    Write-Output $kuduApiAuthorisationToken

    try
    {
        Invoke-RestMethod -Uri $kuduApiUrl `
                        -Headers @{"Authorization"=$kuduApiAuthorisationToken;"If-Match"="*"} `
                        -Method DELETE
    } catch {
        Write-Host "StatusCode:" $_.Exception.Response.StatusCode.value__ 
        Write-Host "StatusDescription:" $_.Exception.Response.StatusDescription
        if (-not ($_.Exception.Response.StatusCode.value__ -eq 404)) {
            throw $PSItem
        }
    }    
}

Delete-KuduFile $resourceGroupName $webAppName $appPath $slotName $kuduPath $recursive

You can then add a Powershell task as mentioned above which should look a little like this... enter image description here

Paul Hatcher
  • 7,342
  • 1
  • 54
  • 51
  • I got this working but in case it helps, I created an Azure PowerShell task and not a regular PowerShell task. The Azure PowerShell task will allow you to get an AzureRM connection and publishing credentials. These are needed to connect to Kudu later on the script. – user3587624 Dec 20 '18 at 11:00
  • @user3587624 Would you be willing to share the modifications you made to the above script for it to intake the AzureRM connection params? – jungos Mar 12 '19 at 22:24
  • 1
    The parameters are the same ones that are above. The only difference is that in your Release Definition, you add an "Azure PowerShell" task instead of a PowerShell task. These two task types are different and the Azure PowerShell task has direct connectivity to AzureRM and it will pick the Release Definition credentials to run the script. – user3587624 Mar 13 '19 at 23:41
  • I've noticed that if you target the root (wwwRoot) with the recursive flag that it deletes the files, but it always throws a 409 Conflict error. This is the same error you would get if you tried to delete a directory without deleting all the files first. I'm assuming that's why I'm getting the 409. I guess I'll just have to catch it as well and just log it to the output so that it doesn't make my CI/CD pipeline fail. – David Yates May 30 '19 at 20:16
0

This is how you can test your settings that you use in the PowerShell scripts above using Postman.

I found it useful to play with the REST API using Postman to understand what the PowerShell scripts above were doing and understand what status codes I could expect back from the API. I also noticed that the recursive logic for deleting a directory returns a 409 Conflict error even though is DOES delete the files.

In the examples below, my app service is called "YatesDeleteMe"

Create a base 64 encoded username and password string to use below in your authorization header OR run the PowerShell scripts above and it will output one for you

  1. Download your app service's publish file, which can be downloaded from the Overview tab in the Azure portal.
  2. Open the file with a text editor
  3. Find the username (user name example: $YatesDeleteMe ) and password (password example: ch222cDlpCjx4Glq333qo4QywGPMs1cK2Rjrn6phqZ9HswtgEEE12CrhDmcn )
  4. Create a string out of them and separate them with a colon (should look something like this: $YatesDeleteMe:ch222cDlpCjx4Glq333qo4QywGPMs1cK2Rjrn6phqZ9HswtgEEE12CrhDmcn )
  5. Base 64 encode them using your own program a site. The result should look something like this: JFlhdGVzRGVsZXRlTWU6Y2gyMjJjRGxwQ2p4NEdscTMzM3FvNFF5d0dQTXMxY0syUmpybjZwaHFaOUhzd3RnRUVFMTJDcmhEbWNu

Retrieve a single file

  1. Open Postman
  2. Change the verb to GET
  3. Enter your URL to the file you want to retrieve (e.g., https://YatesDeleteMe.scm.azurewebsites.net/api/vfs/site/wwwroot/web.config). For more info on paths see this
  4. Added a header-->
    • Key: Authorization
    • Value: Basic YourBase64EncodedStringFromAbove
  5. Left-click the "Send" button and the file will download

Delete a single file

  1. Open Postman
  2. Change the verb to DELETE
  3. Enter your URL to the file you want to delete (e.g., https://YatesDeleteMe.scm.azurewebsites.net/api/vfs/site/wwwroot/web.config)
  4. Added a header-->
    • Key: Authorization
    • Value: Basic YourBase64EncodedStringFromAbove
  5. Added a header-->
    • Key: If-Match
    • Value: *
  6. Left-click the "Send" button and the file will deleted

Delete all the files in a directory

  1. Open Postman
  2. Change the verb to DELETE
  3. Enter your URL to the folder you want to delete PLUST add slash and then the query string ?recursive=true (e.g., https://YatesDeleteMe.scm.azurewebsites.net/api/vfs/site/wwwroot/?recursive=true)
  4. Added a header-->
    • Key: Authorization
    • Value: Basic YourBase64EncodedStringFromAbove
  5. Added a header-->
    • Key: If-Match
    • Value: *
  6. Left-click the "Send" button and the folder will deleted. I always get a 409 error back, but the folder is deleted.

Reference

  • Blog post with pictures here.
David Yates
  • 1,935
  • 2
  • 22
  • 38