2

I was searching for a while for some good design ideas, but haven't found yet the best option. Basically I'm developing serverless API for existing database (RDS MySQL inside VPC, private security group). So I want to query the database from multiple Lambda functions. I know about the following ways how to achieve this:

  1. Place all the Lambdas inside the same VPC. But this case is not good due to slow cold start of Lambdas (I will need to keep them warm) and concurrency limitations. Plus I will need to call another services (S3, SES, third party services) => configuration will be much more difficult. So ideally to keep them outside VPC.
  2. Place a single Lambda (which will only query the DB) inside the same VPC and call it from the other Lambdas outside the VPC using private API Gateway. The Lambda inside VPC will be always warm, but it seems like a bottleneck because of large amount of concurrent requests to this Lambda. I know that it's possible to create multiple security groups and request more IPs, but it will be necessary to monitor the situation constantly, which is not acceptable.
  3. Opening RDS to be accessible from the Internet is not a solution.
  4. Aurora serverless requires VPC as well...

Any ideas or advices?

Thank you!

semenchikus
  • 730
  • 8
  • 17

1 Answers1

2

In answer to the question in your title, no, it is not possible.

From your description, you want to invoke lambdas that can access resources inside and outside of your VPC. Your first two options pretty much describe your only options here.

In regards to your option 1. You can place the lambdas inside the VPC and have a NAT to get back out to the internet. I assume this is what you meant by 'configuration will be much more difficult'. But at least nothing from outside your VPC could initiate connection to something inside the VPC. I don't believe adding a NAT Gateway (see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html) is 'much more difficult', but I guess that is subjective.

In regards to your option 2. You can have a non-VPC Lambda, invoke a Lambda inside a VPC directly (the reverse will not work). So there is no real need for an additional component in the form of an API Gateway (private or otherwise). Lambdas inside VPCs will scale automatically, but limited by the number of available IPs in the sub-net you specify. So some capacity planning would be required.

NB - each option would require at least one lambda in a VPC. Cold starts of these lambdas will incur a slight delay for the ENI (elastic network interface) to be set-up.

K Mo
  • 2,125
  • 8
  • 16
  • Thank you! I guess, option 2 is better way to go in terms of performance, because it should be easier to plan capacity and keep instances of the single VPC Lambda warm. So every other Lambda will query the database through Lambda which is inside VPC. What do you think? – semenchikus Oct 10 '18 at 09:34
  • 1
    With the knowledge that I don't have a clear understanding of how often you are expecting the lambdas to be invoked or what concurrency they may require, I would go with option 2 as it would be the easier to implement and manage in regards to the sub-nets. It would also makes the solution loosely coupled, so swapping out the database in the future would be much simpler (not sure if that is a plus for you or not). – K Mo Oct 10 '18 at 11:47
  • I don't know the exact numbers, but expect hundreds concurrent requests because my serverless API will query DB almost on every request (authentication and different rest requests). As far as I understand, I can request more IPs for the private subnet to increase performance for high-load. Would be better to avoid wrong design from scratch and be ready for high load. – semenchikus Oct 10 '18 at 14:55