1

We have a lambda function in our VPC so that it can connect to our RDS instance. This lambda also needs to connect to s3. It seems that in order to connect to s3 from a VPC, you need to set up a VPC endpoint of the Gateway type. Given the below config we are able to connect to our database, but are still unable to get_object from s3:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.12.0"

  name = var.name
  cidr = var.vpc_cidr

  azs              = ["${var.region}a", "${var.region}b", "${var.region}c"]
  public_subnets   = var.vpc_public_subnets
  private_subnets  = var.vpc_private_subnets
  database_subnets = var.vpc_database_subnets

  create_database_subnet_group       = true
  create_database_subnet_route_table = true

  enable_nat_gateway     = true
  single_nat_gateway     = true
  one_nat_gateway_per_az = false

  enable_dns_hostnames = true

  tags = local.default_tags
}

module "endpoints_us_east_1" {
  source  = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"
  version = "3.10.0"

  vpc_id             = module.vpc.vpc_id
  security_group_ids = [module.security_group_allow_all.security_group_id]

  endpoints = {
    s3 = {
      service         = "s3"
      service_type    = "Gateway"
      route_table_ids = flatten([module.vpc.private_route_table_ids])
      tags            = { Name = "s3-vpc-endpoint" }
    },
  }

  tags = local.default_tags
}

module "security_group_allow_all" {
  source = "terraform-aws-modules/security-group/aws"
  name   = "${var.name}-allow-all"
  vpc_id = module.vpc.vpc_id

  ingress_cidr_blocks = [var.vpc_cidr]
  ingress_rules       = ["all-all"]
  egress_cidr_blocks  = [var.vpc_cidr]
  egress_rules        = ["all-all"]
}

The lambda function (using the terraform module) has these settings applied to it:

  vpc_subnet_ids         = data.terraform_remote_state.foundation.outputs.vpc_private_subnets
  vpc_security_group_ids = [data.terraform_remote_state.foundation.outputs.security_group_allow_all_id]
  attach_network_policy  = true
metersk
  • 11,803
  • 21
  • 63
  • 100
  • 1
    Not sure it is a typo but `.../aws//modules/...` looks like a typo (double forward slash). – jarmod Aug 02 '22 at 15:30
  • 2
    You only need the endpoint gateway if you are in a subnet with no internet access (private subnet without a NAT gateway), although having it saves bandwidth costs regardless. – jordanm Aug 02 '22 at 15:40
  • 1
    Ah, I see that you have enable_dns_hostnames = true and enable_dns_support defaults to true. Looks like you are OK for DNS. – jarmod Aug 02 '22 at 15:41
  • @jarmod i don't believe it is a typo (https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest/submodules/vpc-endpoints) also `tf apply` works fine and generates VPC endpoints. i do not think our private subnet has a NAT gateway, but how can I check to confirm? – metersk Aug 02 '22 at 15:45
  • Do you see any kind of error..? – Paolo Aug 02 '22 at 17:03
  • @Paolo no error, the lambda just hangs at the `s3_client.get_object` portion of my code indefinitely (5 mins, which is the lambda timeout length) – metersk Aug 02 '22 at 17:34
  • Any IAM policies for the Lamda function that might be missing the permission to access the bucket? – Marko E Aug 02 '22 at 18:16
  • @MarkoE it has a policy for full access to all of our buckets. also, i think that would give an access denied error instead of hanging until timeout, with no error message – metersk Aug 02 '22 at 18:55
  • Yeah, you're right. – Marko E Aug 02 '22 at 19:01
  • @metersk and have you defined the endpoint policy? – Marko E Aug 03 '22 at 08:04

0 Answers0