1

TL/DR

Trying to create a Lambda trigger on a AmazonMQ (RabbitMQ) queue, using private subnets and VPC endpoints does not work.

POC Goal

I'm doing this POC: An AmazonMQ (RabbitMQ) in a private subnet and a Lambda triggered by incoming messages to the queue.

Disclaimer

All I'll state here is what I'm learning, any correction will be appreciated.

On networking

Since Amazon MQ is an AWS-managed service, it runs in its own network. So, when we ask AWS to place the broker in a subnet a network interface is created for this broker in the subnet, giving the broker access and reachability in the subnet.

Something similar goes for Lambda, the network interface gives lambda access to the subnet. But to invoke this lambda, since the invoking endpoints live outside our subnet, there is a need of creating a VPC endpoint exposing the lambda endpoints inside the subnet.

The other option is to grant broker with public access (creating public nats) so the broker can reach the public lambda endpoints. enter image description here

The problem

Simply it doesn't work with VPC endpoints option (it does with the public NATs).

Here is the code I'm using: https://gitlab.com/templates14/terraform-templates/-/tree/master/lambda_rabbitmq_trigger

If you want to test just change the AWS account here:

# here using an AWS profile of my own, change it
provider "aws" {
  region                   = "us-east-1"
  profile                  = "myown-terraform"
}

Analysis

As far as I can tell, the broker and lambda have their network interfaces in the same subnet, the security groups are OK (they allow the needed traffic), and the VPC endpoint is created. But the event mapping (aka the-trigger, created manually or using terraform) never can complete the configuration.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
JuanMatias
  • 87
  • 1
  • 1
  • 10
  • 1
    If I'm reading this correctly, the [event source mapping docs](https://docs.aws.amazon.com/lambda/latest/dg/with-mq.html#services-mq-eventsourcemapping) suggest that you must also configure VPC endpoint access to STS and Secrets Manager. – jarmod Feb 27 '22 at 01:01
  • @jarmod thanks, I totally missed that. I will try and come back to state this in the answer. – JuanMatias Feb 28 '22 at 12:25

1 Answers1

1

As @jarmod mentioned (thanks for this), I missed the VPC endpoints for STS and SecretsManager.

Basically, the solution was ok, but this had to be added:

resource "aws_vpc_endpoint" "sts_endpoint" {
  vpc_id            = module.red.vpc_id
  service_name      = "com.amazonaws.${ var.region }.sts"
  vpc_endpoint_type = "Interface"
  subnet_ids = [module.red.private_subnets[0]]
  security_group_ids = [ aws_security_group.sg-endpoint.id ]
  private_dns_enabled = true
}
resource "aws_vpc_endpoint" "secretsmanager_endpoint" {
  vpc_id            = module.red.vpc_id
  service_name      = "com.amazonaws.${ var.region }.secretsmanager"
  vpc_endpoint_type = "Interface"
  subnet_ids = [module.red.private_subnets[0]]
  security_group_ids = [ aws_security_group.sg-endpoint.id ]
  private_dns_enabled = true
}

This is the final diagram:

enter image description here

Here's the code if you want to play with it: https://gitlab.com/templates14/terraform-templates/-/tree/master/lambda_rabbitmq_trigger

JuanMatias
  • 87
  • 1
  • 1
  • 10