0

I am trying to obtain (via terraform) the dns name of a dynamically created VPCE endpoint using a data resource but the problem I am facing is the service name is not known until resources have been created. See notes below.

Is there any way of retrieving this information as a hard-coded service name just doesn’t work for automation?

e.g. this will not work as the service_name is dynamic

    resource "aws_transfer_server" "sftp_lambda" {

      count                  = local.vpc_lambda_enabled
      domain                 = "S3"
      identity_provider_type = "AWS_LAMBDA"
      endpoint_type          = "VPC"
      protocols              = ["SFTP"]
      logging_role           = var.loggingrole
      function               = var.lambda_idp_arn[count.index]

      endpoint_details = {
         security_group_ids = var.securitygroupids
         subnet_ids         = var.subnet_ids
         vpc_id             = var.vpc_id
      }
    
      tags = {
        NAME = "tf-test-transfer-server"
        ENV  = "test"
      } 
    }

    data "aws_vpc_endpoint" "vpce" {
      count        = local.vpc_lambda_enabled
      vpc_id       = var.vpc_id
      service_name = "com.amazonaws.transfer.server.c-001"
      depends_on   = [aws_transfer_server.sftp_lambda]
    }

    output "transfer_server_dnsentry" {
        value = data.aws_vpc_endpoint.vpce.0.dns_entry[0].dns_name
    }

Note: The VPCE was created automatically from an AWS SFTP transfer server resource that was configured with endpoint type of VPC (not VPC_ENDPOINT which is now deprecated). I had no control over the naming of the endpoint service name. It was all created in the background.

Minimum AWS provider version: 3.69.0 required.

Here is an example cloudformation script to setup an SFTP transfer server using Lambda as the IDP. This will create the VPCE automatically. So my aim here is to output the DNS name from the auto-created VPC endpoint using terraform if at all possible.

example setup in cloudFormation

data source: aws_vpc_endpoint resource: aws_transfer_server

Eric Pang
  • 11
  • 4
  • Are you not able to get it from the output of the transfer server resource? – theherk Jul 31 '22 at 09:22
  • No, it doesn’t output the vpc’s dns name nor the Service name. The transfer server outputs the server endpoint but I can’t seem to use that hence why I need the VPCE dns name entry. In order to get this, I need to know the service name after creation. – Eric Pang Jul 31 '22 at 09:30
  • So you want to use the data source to populate the SFTP argument or something else? – Marko E Jul 31 '22 at 09:32
  • I’m using the data source to output the VPC dns name from my transfer server module. So when I deploy the server, I will know what the endpoint is. – Eric Pang Jul 31 '22 at 09:36
  • Not sure I undestand but you don't have to provide both arguments to the data source, you can use only `vpc_id`. – Marko E Jul 31 '22 at 10:12
  • Problem is we have a number of other VPC endpoints within the same VPC so need to select the correct one that belongs to the exact SFTP server. – Eric Pang Jul 31 '22 at 10:16
  • Maybe using filter helps? E.g. if there are specific tags associated with the VPC? – Marko E Jul 31 '22 at 12:12
  • Good suggestion but unfortunately the VPCE was automatically created and so unable to explicitly add any tagging – Eric Pang Jul 31 '22 at 13:13
  • @EricPang You have linked some kind of unofficial AWS provider? Here is the official one and it has what you need I think: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/transfer_server. – Marko E Jul 31 '22 at 18:20
  • sorry @MarkoE I pasted the wrong link - thanks for the spot. I will amend the OP. – Eric Pang Jul 31 '22 at 23:30
  • Ok, so how are you getting that value in the CloudFormation example? And once again, what do you need the VPC Endpoint id for? – Marko E Aug 01 '22 at 13:21
  • The Cloudformation solution doesn't retrieve this info. It is however, the same setup as I have in terraform. I need the endpoint dns name for our users to make the connection to the server or at least refer to this in our Route 53 CNAME record. – Eric Pang Aug 01 '22 at 16:47

2 Answers2

1

I had a response from Hashicorp Terraform Support on this and this is what they suggested:

you can get the service SFTP-Server-created-VPC-Endpoint by calling the following exported attribute of the vpc_endpoint_service resource [a].

NOTE: There are certain setups that causes AWS to create additional resources outside of what you configured. The AWS SFTP transfer service is one of them. This behavior is outside Terraform's control and more due to how AWS designed the service.

You can bring that VPC Endpoint back under Terraform's control however, by importing the VPC endpoint it creates on your behalf AFTER the transfer service has been created - via the VPCe ID [b]. If you want more ideas of pulling the service name from your current AWS setup, feel free to check out this example [c].

Hope that helps! Thank you.

[a] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_service#service_name [b] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint#import [c] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint#gateway-load-balancer-endpoint-type

There is a way forward like I shared earlier with the imports but it not going to be fully automated unfortunately.

Optionally, you can use a provisioner [1] and the aws ec2 describe-vpc-endpoint-services --service-names command [2] to get the service names you need.

I'm afraid that's the last workaround I can provide, as explained in our doc here [3] - which will explain how - as much as we'd like to, Terraform isn't able to solve all use-cases.

[1] https://www.terraform.io/language/resources/provisioners/remote-exec [2] https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/describe-vpc-endpoint-services.html [3] https://www.terraform.io/language/resources/provisioners/syntax

Eric Pang
  • 11
  • 4
0

I've finally found the solution:

data "aws_vpc_endpoint" "transfer_server_vpce" {
  count  = local.is_enabled
  vpc_id = var.vpc_id

  filter {
    name   = "vpc-endpoint-id"
    values = ["${aws_transfer_server.transfer_server[0].endpoint_details[0].vpc_endpoint_id}"]
  }
}
Eric Pang
  • 11
  • 4