Recommendations for AWS CodeStar/Pipeline and Python (Django/Flask)

Question

I am quite new to AWS. I did a course called 'AWS Cloud Practitioner Essentials' and then I have recently been doing a certified developer course which I got from Udemy.

For a while, I have been interested in python, Django and Flask. It was a little bit interesting setting up an EC2 instance, but what really caught my attention was i) setting up autoscaling groups/load balancers. ii) Elastic Beanstalk iii) CodePipeline and iv) CodeStar.

As part of the course, a CodePipeline was setup using Elastic Beanstalk with node.js which created a pre-production environment, manual sign-off and then production release.

Within CodeStar it is pretty easy to setup a Django Project, however I am not very confident about a number of things and was looking for a bit of advice. In particular when a Django Project is setup in CodeStar, instead of creating an Elastic Beanstalk item directly in the CodePipeline, it creates two CloudFormation objects called GenerateChange... and ExecuteChangeSet.

I must admit that I am not confident with CloudFormation. When I look at the Elastic Beanstalk that is generated by the Django Project, it creates a system with one EC2 instance. What I really wanted to try out was a Load Balance/Auto Scaling system. Could you help with the following:

i) If I go directly into Elastic Beanstalk for the Django CodeStar project that was created and change the instance from single to Load Balanced, will this be acceptable or should I try to adjust the CloudFormation settings in the CodePipeline ?

ii) How does the Elastic Beanstalk know how to setup the Django projects on each EC2 instance? When I go into the Elastic Beanstalk project and look at the Configuration section, I cannot see any settings that install any kind of server.

iii) If I was going to add a manual approval and then deployment to production, could I setup a new Elastic Beanstalk instance or would I need to use CloudFormation ?

iv) I haven't got to the bit about adding databases to CodePipeline, but from what I understand, in the production environment, I would need to create a separate database rather than add it straight into Elastic Beanstalk. I wanted to try to get a feel for where I was going. What I was wondering is do you generally automate the migrations by adding them to the buildspec.yml file or do you do something else ?

v) If you have any recommendations for tutorials on this specific area, it would be greatly appreciated.

Thanks for the advice.

Mark

This is a typical buildspec.yml file:

version: 0.2 

phases: 
  install: 
    runtime-versions: 
      python: 3.7 
    commands: 
      # Install dependencies needed for running tests 
      - pip install -r requirements.txt 

  pre_build: 
    commands: 
      # Discover and run unit tests. For more information, see <https://docs.djangoproject.com/en/2.0/topics/testing/overview/> 
      - python manage.py test 

      - aws cloudformation package --template template.yml --s3-bucket $S3_BUCKET --output-template-file template-export.yml 

  post_build: 
    commands: 
      # Do not remove this statement. This command is required for AWS CodeStar projects. 
      # Update the AWS Partition, AWS Region, account ID and project ID in the project ARN on template-configuration.json file so AWS CloudFormation can tag project resources. 
      - sed -i.bak 's/\$PARTITION\$/'${PARTITION}'/g;s/\$AWS_REGION\$/'${AWS_REGION}'/g;s/\$ACCOUNT_ID\$/'${ACCOUNT_ID}'/g;s/\$PROJECT_ID\$/'${PROJECT_ID}'/g' template-configuration.json 
artifacts: 
  type: zip 
  files: 
    - template-export.yml 
    - template-configuration.json 

This the cloudformation template:

AWSTemplateFormatVersion: 2010-09-09
Transform:
- AWS::CodeStar
Conditions:
  UseSubnet:
    Fn::Not:
    - Fn::Equals:
      - Ref: SubnetId
      - subnet-none
Parameters:
  ProjectId:
    Type: String
    Description: AWS CodeStar project ID used to name project resources and create
      roles.
  InstanceType:
    Type: String
    Description: The type of Amazon EC2 Linux instances that will be launched for
      this project.
  KeyPairName:
    Type: String
    Description: The name of an existing Amazon EC2 key pair in the region where the
      project is created, which you can use to SSH into the new Amazon EC2 Linux instances.
  VpcId:
    Type: String
    Description: The ID of the Amazon Virtual Private Cloud (VPC) used for the new
      Amazon EC2 Linux instances.
  SubnetId:
    Type: String
    Description: The name of the VPC subnet used for the new Amazon EC2 Linux instances
      launched for this project.
  SolutionStackName:
    Type: String
    Description: The software stack used to launch environments and configure instances
      in AWS Elastic Beanstalk.
  EBTrustRole:
    Type: String
    Description: The service role in IAM for AWS Elastic Beanstalk to be created for
      this project.
  EBInstanceProfile:
    Type: String
    Description: The IAM role that will be created for the Amazon EC2 Linux instances.
  Stage:
    Type: String
    Description: The name for a project pipeline stage, such as Staging or Prod, for
      which resources are provisioned and deployed.
    Default: ''
Resources:
  EBApplication:
    Description: The AWS Elastic Beanstalk application, which is a container used
      to deploy the correct application configuration.
    Type: AWS::ElasticBeanstalk::Application
    Properties:
      ApplicationName:
        Fn::Sub: ${ProjectId}app${Stage}
      Description: The name of the AWS Elastic Beanstalk application to be created
        for this project.
  EBApplicationVersion:
    Description: The version of the AWS Elastic Beanstalk application to be created
      for this project.
    Type: AWS::ElasticBeanstalk::ApplicationVersion
    Properties:
      ApplicationName:
        Ref: EBApplication
      Description: The application version number.
      SourceBundle:
        S3Bucket: aws-codestar-************-pipe
        S3Key: *************************
  EBConfigurationTemplate:
    Description: The AWS Elastic Beanstalk configuration template to be created for
      this project, which defines configuration settings used to deploy different
      versions of an application.
    Type: AWS::ElasticBeanstalk::ConfigurationTemplate
    Properties:
      ApplicationName:
        Ref: EBApplication
      Description: The name of the sample configuration template.
      OptionSettings:
      - Namespace: aws:elasticbeanstalk:environment
        OptionName: EnvironmentType
        Value: SingleInstance
      - Namespace: aws:elasticbeanstalk:environment
        OptionName: ServiceRole
        Value:
          Ref: EBTrustRole
      - Namespace: aws:elasticbeanstalk:healthreporting:system
        OptionName: SystemType
        Value: enhanced
      SolutionStackName:
        Ref: SolutionStackName
  EBEnvironment:
    Description: The AWS Elastic Beanstalk deployment group where the application
      is deployed, which is made up of the Amazon EC2 Linux instances launched for
      this project.
    Type: AWS::ElasticBeanstalk::Environment
    Properties:
      ApplicationName:
        Ref: EBApplication
      EnvironmentName:
        Ref: EBApplication
      Description: The application to be deployed to the environment.
      TemplateName:
        Ref: EBConfigurationTemplate
      VersionLabel:
        Ref: EBApplicationVersion
      OptionSettings:
      - Namespace: aws:autoscaling:launchconfiguration
        OptionName: IamInstanceProfile
        Value:
          Ref: EBInstanceProfile
      - Namespace: aws:autoscaling:launchconfiguration
        OptionName: InstanceType
        Value:
          Ref: InstanceType
      - Namespace: aws:autoscaling:launchconfiguration
        OptionName: EC2KeyName
        Value:
          Ref: KeyPairName
      - Namespace: aws:ec2:vpc
        OptionName: VPCId
        Value:
          Ref: VpcId
      - Fn::If:
        - UseSubnet
        - Namespace: aws:ec2:vpc
          OptionName: Subnets
          Value:
            Ref: SubnetId
        - Ref: AWS::NoValue

These are snapshots of the CodePipeline Configurations for CloudFormation:

Answer 1

Well that was very big question. I will try to explain.

i) Yes you can change single into loadbalanced. Only thing you need to consider is type of load balancer (classic, application, network) once you selected the type of loadbalancer you cannot change it's type but you can rollback to single instance.

ii) Elastic beanstalk uses auto scaling launch configuration to setup the project on the EC2 instances you cannot see those configurations in the Beanstalk console. It is something that Beanstalk takes care it. I hope will you will get some idea when you visit the autoscaling launch configuration console.

iii) I am not sure what you are pointing to. Manual approval is something you can setup in the existing codepipeline.

iv) It is recommended to create RDS separately instead of in the beanstalk environment because incase if you terminate the environment the associated RDS will also be terminated we don't want to do that.

v) There is a lot tutorials and youtube videos available online. I will suggest ACloud Guru tutorials but it is paid.