I am currently setting up AWS MWAA. I will be using Apache Airflow for performing aggregations on an AWS RDS database that is in the default VPC. Should I create a new VPC for AWS MWAA or can I use the default VPC? What are the advantages of having a separate VPC?
AWS documentation mostly describes what to do when you want to create a new VPC for MWAA.
A relevant section from the Get Started guide says:
You can configure an existing VPC that meets these requirements (advanced) as seen in About networking on Amazon MWAA, or create the VPC and networking components, as defined in Create the VPC network.
It looks like both using an existing VPC and creating a new one are supported options.
The same document says that VPCs shared between AWS accounts are not supported. That might be a reason to create a new VPC.
As far as I can tell, having a separate VPC is popular with Ops people because then services are properly separated from one another, with interactions between services being properly managed with proper oversight.
But the nature of Airflow is to talk to lots of different services in your AWS account (RDS databases, S3 buckets etc.) so perhaps having a single VPC makes life easier.
Disclaimer: this isn't at all a recommendation. I'm a software engineer not an ops person and I don't have any insight at all into whether or not putting everything into one VPC is a good idea.
You can create your own VPC and use it for MWAA. There's nothing special about the one created by default. It is a CloudFormation stack that makes a VPC, some subnets, security groups, routing tables, etc. Your VPC will simply have to provide equivalents of these. It will be more work to create and maintain your own VPC.
However, you say that you want to do this because you are trying to access an RDS from Airflow. It is irrelevant what VPC the target resource is. Airflow can connect to other VPCs, other AWS accounts, even other platforms like Azure or GCP. Remember that Airflow itself is not AWS specific, it is a generic open source program that can be installed on any type of server.
You will simply have to configure an AWS connection for the RDS you want to access, and put in things like its URL, username and password there. Presumably you can connect to this RDS from your own computer. Airflow can do the same. You want to check the access settings of the RDS, in case there's something like blocking all IPs by default but allowing your PC's IP through - you'll obviously have to add Airflow's IP to the exceptions in that case.
