Azure ARM uniqueString function mimic

Question

I need to deploy Sql Databases into an Azure Sql Server using to ways: the ARM template way, and a more custom way using C# code. There's a ARM template function called uniqueString(string) that generate a pseudo random hash of a given string. It's a deterministic pure function.

I need to find a way to exactly mimic the behaviour of this function from my C# code. ie I need to reproduce this function into my C# code.

Where can i find the algorithm used by the ARM Api ?

MSDN reference for uniqueString()

Answer 1

Update 2023-01-05 - As suggested by other answers, there's an easier way now - just reference the Azure.Deployments.Expression nuget package which contains all of the Arm functions and then use the following convenience wrapper:

using Azure.Deployments.Expression.Expressions;
using Newtonsoft.Json.Linq;

public static class ArmFunctions
{
    public static string? UniqueString(params string[] values)
    {
        var parameters = values.Select(
            arg => new FunctionArgument(
                JToken.FromObject(arg)
            )
        ).ToArray();
        var result = ExpressionBuiltInFunctions.Functions
            .EvaluateFunction("uniqueString", parameters, null);
        return result.Value<string>();
    }
}

// "zcztcwvu6iyg6"
var unique = ArmFunctions.UniqueString("tyeth");

---

Original answer for posterity:

I've been researching this myself on and off for a few years now, and I've finally hit paydirt...

// "zcztcwvu6iyg6"
var unique = ArmUniqueString("tyeth");

My ArmUniqueString function is a wrapper around some dlls that are distributed with the Azure Stack Hub Development Kit which is basically a virtual machine image that contains the Azure server-side platform that you can run locally...

private static string ArmUniqueString(string originalString)
{

    var assembly = Assembly.GetAssembly(
        typeof(Microsoft.WindowsAzure.ResourceStack.Frontdoor.Templates.Engines.TemplateEngine)
    );

    var functions = assembly.GetType(
        "Microsoft.WindowsAzure.ResourceStack.Frontdoor.Templates.Expressions.TemplateExpressionBuiltInFunctions"
    );

    var uniqueString = functions.GetMethod(
        "UniqueString",
        BindingFlags.Static | BindingFlags.NonPublic
    );

    var parameters = new object[] {
        "uniqueString",
        new JToken[] {
            (JToken)originalString
        }
    };

    var result = uniqueString.Invoke(null, parameters).ToString();

    return result;

}

You'll need to download the Azure Stack Hub Development Kit and unpack it to get the dlls:

• Download the Azure Stack Hub Development Kit - warning: it's about 22Gb!
• Run the installer to unpack a 55Gb *.vhdx
• Mount the *.vhdx, or expand / unpack it locally
• Inside the *.vhdx, find this file and unzip it somewhere:
  • CloudBuilder\CloudDeployment\NuGetStore\Microsoft.AzureStack.Setup.Services.ResourceManager.5.20.1335.300.nupkg
• The content\Website\bin folder inside the *.nupkg contains the necessary dlls

To use them, add an assembly reference to Microsoft.WindowsAzure.ResourceStack.Frontdoor.Templates.dll (it has some dependencies on other files in the bin folder) and that contains the TemplateExpressionBuiltInFunctions class. The code above just uses reflection to invoke the private UniqueString function from that assembly, with a little bit of work to marshal the parameters into appropriate JToken types.

If you wanted to dig into the implementation details you could probably run a decompiler against the assembly to find out what it's doing under the covers...

Note - credits go to this blog article for pointing me in the right direction:

https://the.agilesql.club/2017/12/azure-arm-template-function-internals/

Answer 2

I found some PowerShell code to do this here: https://blogs.technet.microsoft.com/389thoughts/2017/12/23/get-uniquestring-generate-unique-id-for-azure-deployments/

I converted this code to C#:

public string GetUniqueString(string id, int length = 13)
{
    string result = "";
    var buffer = System.Text.Encoding.UTF8.GetBytes(id);
    var hashArray = new System.Security.Cryptography.SHA512Managed().ComputeHash(buffer);
    for(int i = 1; i <= length; i++)
    {
        var b = hashArray[i];
        var c = Convert.ToChar((b % 26) + (byte)'a');
        result = result + c;
    }

    return result;
}

Answer 3

This function is released in nuget: https://www.nuget.org/packages/Azure.Deployments.Expression/ The implementation: https://msazure.visualstudio.com/One/_git/AzureUX-Deployments?path=%2Fsrc%2FExpressions%2FExpressions%2FExpressionBuiltInFunctions.cs&version=GBmaster&_a=contents

Example:

using Azure.Deployments.Expression.Expressions;
using Newtonsoft.Json.Linq;

var funcs = ExpressionBuiltInFunctions.Functions;
var jt = new JTokenExpression("test");
var output = funcs.EvaluateFunction("uniqueString", new JToken[] { jt.Value }).ToString();

Answer 4

Sam Cogan wrote a blog post on how to do this in C# here: https://samcogan.com/using-bicep-functions-in-c-if-you-really-want-to/

Inspired by Sam, I wrote a PowerShell module (for PowerShell 7) that does the same. You can install the module by running Install-Module -Name AzExpression which will give you command called New-AzUniqueString.

Here is an example on how to use it:
New-AzUniqueString -InputStrings 'test', 'value'
Which will output: bhxq2thzm5dym

Answer 5

Here's a self-contained solution.

static class AzureExtensions
{
    private static string Base32Encode(this ulong value) {
        const string alphabet = "abcdefghijklmnopqrstuvwxyz234567";

        var stringBuilder = new StringBuilder();

        for (var index = 0; (index < 13); ++index) {
            _ = stringBuilder.Append(value: alphabet[((int)(value >> 59))]);
            value <<= 5;
        }

        return stringBuilder.ToString();
    }
    private static ulong MurmurHash64(this byte[] data, uint seed = 0) {
        const uint magic0 = 2869860233U;
        const uint magic1 = 0597399067U;
        const uint magic2 = 2246822507U;
        const uint magic3 = 3266489909U;

        var hash0 = seed;
        var hash1 = seed;
        var index = 0;
        var length = data.Length;

        while ((index + 7) < length) {
            var temp0 = ((uint)(data[index] | (data[(index + 1)] << 8) | (data[(index + 2)] << 16) | (data[(index + 3)] << 24)));
            var temp1 = ((uint)(data[(index + 4)] | (data[(index + 5)] << 8) | (data[(index + 6)] << 16) | (data[(index + 7)] << 24)));
            var temp2 = (uint.RotateLeft(value: (temp0 * magic1), rotateAmount: 15) * magic0);
            var temp3 = (uint.RotateLeft(value: (temp1 * magic0), rotateAmount: 17) * magic1);

            hash0 = ((uint)((((int)(uint.RotateLeft(rotateAmount: 19, value: (hash0 ^ temp2)) + hash1)) * 5) + 1444728091));
            hash1 = ((uint)((((int)(uint.RotateLeft(rotateAmount: 13, value: (hash1 ^ temp3)) + hash0)) * 5) + 197830471));
            index += 8;
        }

        var remainder = (length - index);

        if (0 < remainder) {
            int temp0;

            if (4 > remainder) {
                temp0 = remainder switch {
                    2 => (data[index] | (data[(index + 1)] << 8)),
                    3 => (data[index] | (data[(index + 1)] << 8) | (data[(index + 2)] << 16)),
                    _ => data[index],
                };
            }
            else {
                temp0 = (data[index] | (data[(index + 1)] << 8) | (data[(index + 2)] << 16) | (data[(index + 3)] << 24));
            }

            hash0 ^= (uint.RotateLeft(value: (((uint)temp0) * magic1), rotateAmount: 15) * magic0);

            if (4 < remainder) {
                temp0 = remainder switch {
                    6 => (data[(index + 4)] | (data[(index + 5)] << 8)),
                    7 => (data[(index + 4)] | (data[(index + 5)] << 8) | (data[(index + 6)] << 16)),
                    _ => data[index + 4],
                };
                hash1 ^= (uint.RotateLeft(value: (((uint)temp0) * magic0), rotateAmount: 17) * magic1);
            }
        }

        var result0 = (hash1 ^ ((uint)length));
        var result1 = ((hash0 ^ ((uint)length)) + result0);
        var result2 = (result0 + result1);
        var result3 = ((result1 ^ (result1 >> 16)) * magic2);
        var result4 = ((result3 ^ (result3 >> 13)) * magic3);
        var result5 = ((result2 ^ (result2 >> 16)) * magic2);
        var result6 = ((result5 ^ (result5 >> 13)) * magic3);
        var result7 = (result6 ^ (result6 >> 16));
        var result8 = ((result4 ^ (result4 >> 16)) + result7);

        return ((((ulong)(result7 + result8)) << 32) | result8);
    }

    public static string UniqueString(this string value) =>
        Encoding.UTF8.GetBytes(s: value).MurmurHash64().Base32Encode();
    public static string UniqueString(this IEnumerable<string> values) =>
        Encoding.UTF8.GetBytes(s: string.Join(separator: '-', values: values)).MurmurHash64().Base32Encode();
}

Answer 6

I finally found a workaround. I used a very simple ARM template which goal is to only output the result of the uniqueStringcommand. Then I fetch this output in my C# code. This solution is not really the quickest one ;-), but it works as desired.

Answer 7

Here is a bicep template which will output a uniqueString for a resource group:

output unique string = uniqueString(resourceGroup().id)