How to setup private-link using Terraform to access storage-account?

Question

I need to test my azure private-endpoint using the following scenario.

We have a virtual netwroks with two sub-nets (vm_subnet and storage_account_subnet)
The virtual-machine (vm) should be able to connect to the storage-account using a private-link.

So, the below image explains the scenario:

and then i need to test my endpoint using the below manual test-case:

Connect to the azure virtual-machine using ssh and putty username: adminuser and password: P@$$w0rd1234!
In the terminal ping formuleinsstorage.blob.core.windows.net (Expect to see the ip of storage account in the range of storage_account_subnet (10.0.2.0/24))

I deploy all the infrastructure using the below Terraform code:

provider "azurerm" {

  features {
    resource_group {
      prevent_deletion_if_contains_resources = false
    }
  }
}

resource "azurerm_resource_group" "main_resource_group" {
  name     = "RG-Terraform-on-Azure"
  location = "West Europe"
}
# Create Virtual-Network

resource "azurerm_virtual_network" "virtual_network" {
  name                = "Vnet"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name
}


# Create subnet for virtual-machine

resource "azurerm_subnet" "virtual_network_subnet" {
  name                 = "vm_subnet"
  resource_group_name  = azurerm_resource_group.main_resource_group.name
  virtual_network_name = azurerm_virtual_network.virtual_network.name
  address_prefixes     = ["10.0.1.0/24"]
}

# Create subnet for storage account

resource "azurerm_subnet" "storage_account_subnet" {
  name                 = "storage_account_subnet"
  resource_group_name  = azurerm_resource_group.main_resource_group.name
  virtual_network_name = azurerm_virtual_network.virtual_network.name
  address_prefixes     = ["10.0.2.0/24"]
}

# Create Linux Virtual machine
resource "azurerm_linux_virtual_machine" "example" {
  name                            = "example-machine"
  location                        = azurerm_resource_group.main_resource_group.location
  resource_group_name             = azurerm_resource_group.main_resource_group.name
  size                            = "Standard_F2"
  admin_username                  = "adminuser"
  admin_password                  = "14394Las?"
  disable_password_authentication = false
  network_interface_ids           = [
    azurerm_network_interface.virtual_machine_network_interface.id,
  ]


  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }
}

resource "azurerm_network_interface" "virtual_machine_network_interface" {
  name                = "vm-nic"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_subnet.virtual_network_subnet.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.vm_public_ip.id
  }
}

# Create Network-interface and public-ip for virtual-machien

resource "azurerm_public_ip" "vm_public_ip" {
  name                = "vm-public-ip-for-rdp"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name
  allocation_method   = "Static"
  sku                 = "Standard"
}

resource "azurerm_network_interface" "virtual_network_nic" {
  name                = "vm_nic"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name

  ip_configuration {
    name                          = "testconfiguration1"
    subnet_id                     = azurerm_subnet.virtual_network_subnet.id
    private_ip_address_allocation = "Dynamic"
  }
}

# Setup an Inbound rule because we need to connect to the virtual-machine using RDP (remote-desktop-protocol)

resource "azurerm_network_security_group" "traffic_rules" {
  name                = "vm_traffic_rules"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name

  security_rule {
    name                       = "virtual_network_permission"
    priority                   = 100
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "*"
    source_port_range          = "*"
    destination_port_range     = "22"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
}

resource "azurerm_subnet_network_security_group_association" "private_nsg_asso" {
  subnet_id                 = azurerm_subnet.virtual_network_subnet.id
  network_security_group_id = azurerm_network_security_group.traffic_rules.id

}

# Setup storage_account and its container

resource "azurerm_storage_account" "storage_account" {
  name                     = "storagaccountfortest"
  location                 = azurerm_resource_group.main_resource_group.location
  resource_group_name      = azurerm_resource_group.main_resource_group.name
  account_tier             = "Standard"
  account_replication_type = "LRS"
  account_kind             = "StorageV2"
  is_hns_enabled           = "true"

}

resource "azurerm_storage_data_lake_gen2_filesystem" "data_lake_storage" {
  name               = "rawdata"
  storage_account_id = azurerm_storage_account.storage_account.id

  lifecycle {
    prevent_destroy = false
  }
}

# Setup DNS zone
resource "azurerm_private_dns_zone" "dns_zone" {
  name                = "privatelink.blob.core.windows.net"
  resource_group_name = azurerm_resource_group.main_resource_group.name
}

resource "azurerm_private_dns_zone_virtual_network_link" "network_link" {
  name                  = "vnet_link"
  resource_group_name   = azurerm_resource_group.main_resource_group.name
  private_dns_zone_name = azurerm_private_dns_zone.dns_zone.name
  virtual_network_id    = azurerm_virtual_network.virtual_network.id

}

# Setup private-link

resource "azurerm_private_endpoint" "endpoint" {
  name                = "storage-private-endpoint"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name
  subnet_id           = azurerm_subnet.storage_account_subnet.id

  private_service_connection {
    name                           = "private-service-connection"
    private_connection_resource_id = azurerm_storage_account.storage_account.id
    is_manual_connection           = false
    subresource_names              = ["blob"]
  }
}

resource "azurerm_private_dns_a_record" "dns_a" {
  name                = "dns-record"
  zone_name           = azurerm_private_dns_zone.dns_zone.name
  resource_group_name = azurerm_resource_group.main_resource_group.name
  ttl                 = 10
  records             = [azurerm_private_endpoint.endpoint.private_service_connection.0.private_ip_address]
}

The problem is, my endpoint does not work ! But, if i try to add the service-endpoint manually then everything works well like a charm. So,i think, my DNS Zone is correct, also apparently the link to the storage-account is also working well. So, I think there should be something wrong with my private-link ! Any idea ?

Update:

Here are versions:

Terraform v1.2.5
on windows_386
+ provider registry.terraform.io/hashicorp/azurerm v3.30.0

Which provider version are you using? – Marko E Nov 09 '22 at 22:12 — Marko E, Nov 09 '22 at 22:12

score 3 · Accepted Answer · answered Nov 16 '22 at 06:53

I believe the issue lies in the name of the dns_a_record. This should be the name of the storage account you want to reach via the private link.

The following Terraform code is working for me:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.30.0"
    }
  }
}

provider "azurerm" {
  features {
    resource_group {
      prevent_deletion_if_contains_resources = false
    }
  }
}

resource "azurerm_resource_group" "main_resource_group" {
  name     = "RG-Terraform-on-Azure"
  location = "West Europe"
}


# Create Virtual-Network
resource "azurerm_virtual_network" "virtual_network" {
  name                = "Vnet"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name
}


# Create subnet for virtual-machine
resource "azurerm_subnet" "virtual_network_subnet" {
  name                 = "vm_subnet"
  resource_group_name  = azurerm_resource_group.main_resource_group.name
  virtual_network_name = azurerm_virtual_network.virtual_network.name
  address_prefixes     = ["10.0.1.0/24"]
}

# Create subnet for storage account
resource "azurerm_subnet" "storage_account_subnet" {
  name                 = "storage_account_subnet"
  resource_group_name  = azurerm_resource_group.main_resource_group.name
  virtual_network_name = azurerm_virtual_network.virtual_network.name
  address_prefixes     = ["10.0.2.0/24"]
}

# Create Linux Virtual machine
resource "azurerm_linux_virtual_machine" "example" {
  name                            = "example-machine"
  location                        = azurerm_resource_group.main_resource_group.location
  resource_group_name             = azurerm_resource_group.main_resource_group.name
  size                            = "Standard_F2"
  admin_username                  = "adminuser"
  admin_password                  = "14394Las?"
  disable_password_authentication = false
  network_interface_ids           = [
    azurerm_network_interface.virtual_machine_network_interface.id,
  ]


  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }
}

resource "azurerm_network_interface" "virtual_machine_network_interface" {
  name                = "vm-nic"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name

  ip_configuration {
    name                          = "internal"
    subnet_id                     = azurerm_subnet.virtual_network_subnet.id
    private_ip_address_allocation = "Dynamic"
    public_ip_address_id          = azurerm_public_ip.vm_public_ip.id
  }
}

# Create Network-interface and public-ip for virtual-machien
resource "azurerm_public_ip" "vm_public_ip" {
  name                = "vm-public-ip-for-rdp"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name
  allocation_method   = "Static"
  sku                 = "Standard"
}

resource "azurerm_network_interface" "virtual_network_nic" {
  name                = "storage-private-endpoint-nic"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name

  ip_configuration {
    name                          = "storage-private-endpoint-ip-config"
    subnet_id                     = azurerm_subnet.virtual_network_subnet.id
    private_ip_address_allocation = "Dynamic"
  }
}

# Setup an Inbound rule because we need to connect to the virtual-machine using RDP (remote-desktop-protocol)
resource "azurerm_network_security_group" "traffic_rules" {
  name                = "vm_traffic_rules"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name

  security_rule {
    name                       = "virtual_network_permission"
    priority                   = 100
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "*"
    source_port_range          = "*"
    destination_port_range     = "22"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
}

resource "azurerm_subnet_network_security_group_association" "private_nsg_asso" {
  subnet_id                 = azurerm_subnet.virtual_network_subnet.id
  network_security_group_id = azurerm_network_security_group.traffic_rules.id

}

# Setup storage_account and its container
resource "azurerm_storage_account" "storage_account" {
  name                     = <STORAGE_ACCOUNT_NAME>
  location                 = azurerm_resource_group.main_resource_group.location
  resource_group_name      = azurerm_resource_group.main_resource_group.name
  account_tier             = "Standard"
  account_replication_type = "LRS"
  account_kind             = "StorageV2"
  is_hns_enabled           = "true"

}

resource "azurerm_storage_data_lake_gen2_filesystem" "data_lake_storage" {
  name               = "rawdata"
  storage_account_id = azurerm_storage_account.storage_account.id

  lifecycle {
    prevent_destroy = false
  }
}

# Setup DNS zone
resource "azurerm_private_dns_zone" "dns_zone" {
  name                = "privatelink.blob.core.windows.net"
  resource_group_name = azurerm_resource_group.main_resource_group.name
}

resource "azurerm_private_dns_zone_virtual_network_link" "network_link" {
  name                  = "vnet-link"
  resource_group_name   = azurerm_resource_group.main_resource_group.name
  private_dns_zone_name = azurerm_private_dns_zone.dns_zone.name
  virtual_network_id    = azurerm_virtual_network.virtual_network.id
}

# Setup private-link
resource "azurerm_private_endpoint" "endpoint" {
  name                = "storage-private-endpoint"
  location            = azurerm_resource_group.main_resource_group.location
  resource_group_name = azurerm_resource_group.main_resource_group.name
  subnet_id           = azurerm_subnet.storage_account_subnet.id

  private_service_connection {
    name                           = "storage-private-service-connection"
    private_connection_resource_id = azurerm_storage_account.storage_account.id
    is_manual_connection           = false
    subresource_names              = ["blob"]
  }
}

resource "azurerm_private_dns_a_record" "dns_a" {
  name                = azurerm_storage_account.storage_account.name
  zone_name           = azurerm_private_dns_zone.dns_zone.name
  resource_group_name = azurerm_resource_group.main_resource_group.name
  ttl                 = 10
  records             = [azurerm_private_endpoint.endpoint.private_service_connection.0.private_ip_address]
}

Additionally, I'm not sure whether it is possible to ping storage accounts. To test I ran nslookup <STORAGE_ACCOUNT_NAME>.blob.core.windows.net both from my local machine and from the Azure VM. In the former case, I got a public IP while in the latter I got a private IP in the range defined in the Terraform config, which seems to be the behaviour you are looking for.

Thanks for the response. why do you think `ping` is not the good test ? — Jeff, Nov 16 '22 at 08:55
It wasn't working for me and a brief search revealed that this is ([or at least was in the past](https://social.msdn.microsoft.com/Forums/en-US/062b3a5e-96fd-4d15-be29-4b847844bd5f/file-storage-no-ping-possible?forum=windowsazuredata)) not possible. — matsu, Nov 16 '22 at 09:01
I got you. so, now if i want to limit acces to the storage-account, i need to apply network_security_group both to the endpoint and the storage_account ? Am i right ? — Jeff, Nov 16 '22 at 09:29
There are several ways. You can use the [network_rules property](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/storage_account#network_rules) to allow only access from certain IPs or virtual subnets. Since a private endpoint is already in place, you can also set the [public_network_access_enabled property](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/storage_account#public_network_access_enabled) to false. In this case access is only possible via the private endpoint. — matsu, Nov 16 '22 at 12:02

How to setup private-link using Terraform to access storage-account?

1 Answers1