0

I had an ecs cluster running with ec2: I had a service running a nginx task and i had an ec2 autoscaling group, with an ALB in front of then. The task network interface was awsvcp. it worked fine but, as i need to allow dynamic port mapping (for running more than one task per ec2 instance), i changed my settings (now the task uses a bridge interface network and allow dynamic port mapping (host port = 0)). But, since i did that changes, my alb receive 504 (timeout) when try to communicate with the ec2 instances. And i can't even ssh inside the ec2 instance anymore (timeout too). Why this small setting (changing the network interface for dynamic port mapping) messed up my cluster? I suspect is something related to the ec2 instances settings, because i can't even ssh on then anymore. Bellow i pasted the keys settings on my cloudformation template:

   LoadBalancer:
    Condition: CreateMainResources
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties: 
      Scheme: internet-facing
      Subnets:
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      Type: application
      SecurityGroups:
        - !Ref ECSSecurityGroup

  Listener80:
    Condition: CreateMainResources
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: !Ref LoadBalancer
      Port: !Ref ListeningOnPort 
      Protocol: HTTP
      DefaultActions:
        - TargetGroupArn: !Ref MyTargetGroup
          Type: forward
  
  MyTargetGroup:
    Condition: CreateMainResources
    DependsOn: LoadBalancer
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Matcher:
        HttpCode: 200-499  # 200-499 ou 200,204
      Port: !Ref ListeningOnPort 
      Protocol: HTTP
      TargetType: instance # ip
      VpcId: !Ref VPC

  EC2LaunchTemplate:
      Type: AWS::EC2::LaunchTemplate
      Condition: CreateEC2Resources
      Properties: 
        LaunchTemplateData:
          ImageId: !Ref Ec2ImageId
          InstanceType: !Ref InstanceType
          IamInstanceProfile:
            Arn: !GetAtt EC2InstanceProfile.Arn
          Monitoring: 
            Enabled: true
          KeyName: !Ref Key
          NetworkInterfaces: 
            - AssociatePublicIpAddress: true
              DeviceIndex: '0'
              Groups: 
                - !GetAtt EC2SecurityGroup.GroupId
              SubnetId: !Ref PublicSubnet1
          UserData:
            Fn::Base64: !Sub 
              - |
                #!/bin/bash
                echo ECS_CLUSTER=${cluster_name} >> /etc/ecs/ecs.config
              - cluster_name: !Sub ${AWS::StackName}-cluster 
    
  EC2SecurityGroup:
        Type: "AWS::EC2::SecurityGroup"
        Properties:
          VpcId: !Ref VPC
          SecurityGroupIngress: 
            - CidrIp: 0.0.0.0/0
              FromPort: !Ref ListeningOnPort 
              IpProtocol: "tcp"
              ToPort: !Ref ListeningOnPort 
            - IpProtocol: tcp
              FromPort: 22
              ToPort: 22
              CidrIp: !Ref SSHUserIP
         
  NginxWebServerTaskDefinition: 
      Condition: CreateECSResources
      Type: AWS::ECS::TaskDefinition
      Properties:
        ContainerDefinitions: 
          - Name: !Ref TaskContainerName
            Image: !Ref ContainerDefinitionImage
            Essential: true
            Privileged: false
            PortMappings:
              - ContainerPort: !Ref ListeningOnPort 
                HostPort: 0 # !Ref ListeningOnPort
                Protocol: tcp
            LogConfiguration:
              LogDriver: awslogs
              Options:
                awslogs-group: !Ref LogGroup
                awslogs-region: us-east-1
                awslogs-stream-prefix: nginx
        Cpu: !Ref TaskDefinitionCpu 
        Memory: !Ref TaskDefinitionMemory
        ExecutionRoleArn: !Ref ExecutionRole
        Family: !Sub ${AWS::StackName}-nginx-task
        NetworkMode: bridge # awsvpc  
        RequiresCompatibilities: 
          - EC2
        TaskRoleArn: !Ref TaskRole
    
    
  ECSSecurityGroup:
      Condition: CreateMainResources
      Type: AWS::EC2::SecurityGroup
      Properties:
        SecurityGroupIngress:
          - IpProtocol: tcp
            FromPort: !Ref ListeningOnPort 
            ToPort: !Ref ListeningOnPort 
            CidrIp: 0.0.0.0/0
        VpcId: !Ref VPC
    
  Service:
      Condition: CreateECSResources
      DependsOn:
        - Listener80
        - EC2AutoScalingGroup
      Type: AWS::ECS::Service
      Properties:
        Cluster: !Ref Cluster 
        CapacityProviderStrategy:
          - CapacityProvider: !Ref MainCapacityProvider
            Weight: !Ref Weight
        TaskDefinition: !Ref NginxWebServerTaskDefinition 
        SchedulingStrategy: REPLICA 
        DeploymentConfiguration:
          MaximumPercent: 200
          MinimumHealthyPercent: 100
        DeploymentController:
          Type: ECS
        PlacementStrategies:
          - Type: binpack 
            Field: memory
        DesiredCount: !Ref TaskDefinitionInstantiations
        LoadBalancers: 
          - ContainerName: !Ref TaskContainerName
            ContainerPort: !Ref ListeningOnPort
            TargetGroupArn: !Ref MyTargetGroup
        # NetworkConfiguration: # awsvpc only
        #   AwsvpcConfiguration:
        #     Subnets: 
        #       - !Ref PublicSubnet1
        #       - !Ref PublicSubnet2
        #     SecurityGroups:
        #       - !Ref ECSSecurityGroup
victor israe
  • 415
  • 3
  • 14

1 Answers1

0

The issue was that my ec2 instance must listening on all possible ephemeral host port to it works (because of the dynamic mapping setting), otherwise those port was unreachable and the timeout would be trigger. So, i needed to change my security group settings:

EC2SecurityGroup:
    Type: "AWS::EC2::SecurityGroup"
    Properties:
      VpcId: !Ref VPC
      SecurityGroupIngress: 
        - IpProtocol: tcp
          # range de portas efemeras do alb 32768-65535 
          FromPort: !If [DynamicPortMapping, 32768, !Ref ListeningOnPort]
          ToPort: !If [DynamicPortMapping, 65535, !Ref ListeningOnPort ]
          SourceSecurityGroupId: !Ref ECSSecurityGroup
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: !Ref SSHUserIP

references:

https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_PortMapping.html

https://aws.amazon.com/premiumsupport/knowledge-center/dynamic-port-mapping-ecs/

https://www.youtube.com/watch?v=cmRZleI18Yg ( 4:52 - 5:12 is the key moment)

victor israe
  • 415
  • 3
  • 14