0

I was performing AWS Blue/Green deployment to test a basic apache server and as CodeDeploy goes through 4 steps of B/G deployment it stuck at step 3 i.e, Routing traffic to replacement instances for about half-hour. I am not able to figure out what's wrong even after spending a long time and a couple of retries. I don't know whether my setup is correct or not. Here is my setup for B/G deployment.

  1. Created an autoscaling group named orginalautoscalinggroup with two instances.
  2. Creted two target groups one for original env and second for replacement env namely originaltargetgroup & replacementtargetgroup.
  3. Attached originaltargetgroup to be a part of originalautoscalinggroup.
  4. Created an application load balancer namely apploadbalancer and attached it with originaltargetgroup
  5. Now apploadbalancer is attached with originaltargetgroup which is attached with the originalautoscalinggroup.

My Codedeploy B/G configuration looks like this.

  1. Deployment Type: B/G
  2. Env configuration: Automatically copy Amazon EC2 Auto Scaling group
  3. Amazon EC2 Autoscaling group: originalautoscalinggroup
  4. Traffic rerouting: Reroute traffic immediately
  5. Termination policy: Terminate the original instances in the deployment group
  6. Deployment configuration: CodeDeployDefault.AllAtOnce
  7. Application load balancer.
  8. Target group: originaltargetgroup

I think I am not able to attach correct load balancer or target group. Please help.

Image1

Image2

marshmello
  • 71
  • 4
  • 10
  • Can you post screenshots of CodeDeploy deployment where it is stuck? Also, is it stuck in rerouting the traffic or is it struck after traffic is rerouted? – saurabh14292 Apr 11 '20 at 13:57
  • @saurabh14292 I have added two images Image1 & Image2 please check it out. Also, it got stuck at Allow Traffic so I stopped the deployment. Please help as I have spent a long time trying out B/G deployment but couldn't get the right way. – marshmello Apr 13 '20 at 05:52
  • Have you checked if 1. New instances span up because of new ASG are registered to TargetGroup of LoadBalancer? 2. New instances are in "healthy" state so that they can server the traffic. – saurabh14292 Apr 13 '20 at 11:41
  • @saurabh14292 Whenever I create an autoscaling group it launches two instances [empty] not reachable by their ip's so they are marked unhealthy by the target groups and I am trying to install the application on these fresh instances and maybe because instances are unhealthy (due to their success codes) It got stuck at allowtraffic showing error instances are unhealthy. So how do I cope with the success code and healthy state of instances? – marshmello Apr 13 '20 at 13:03
  • So my question is how do I make fresh instances reachable as they will always remain unhealthy because nothing is configured on them to make them reachable. – marshmello Apr 13 '20 at 13:10
  • You will have to configure your ASG with Launch Template in such a way that, once EC2 is started, it will install, configure and start the required application. Say Nginx running on port 80 or tomcat running on 8080. This can be done using "User Data" where you can write down commands/upload script to run required steps. – saurabh14292 Apr 13 '20 at 13:39
  • Or create a custom AMI with your application baked, and required services to start on boot. – saurabh14292 Apr 13 '20 at 13:40

1 Answers1

0

Adding answer based on comments.

Since the B/G deployment is failing during traffic rerouting, specifically at "Allow Traffic", instances are in "Unhealthy" state.

If the required services are not running on EC2 and are not reachable, they will remain in unhealthy state and traffic cannot be redirected to them, hence deployment will fail.

To make sure required services/processes are running on EC2 servers started by AutoScaling group, you

  1. Update the Launch Config/Template with "User Data" to execute the script on EC2 startup, which will install, configure and start the required services, say Nginx on port 80.
  2. You can create a custom AMI which has required application already installed in it and on EC2 startup, required service/process will start.

This way, whenever a new EC2 is started by ASG, it will go into healthy state depending on time required to start the process. Once it is healthy, it will be ready to server the traffic and post that, old instances will get de-registered.

Once old instances are de-registered, they may get terminated based on configurations in CodeDeploy.

saurabh14292
  • 1,281
  • 2
  • 10
  • 12
  • I have launched new instances in the autoscaling group configured with the apache webserver running on port 80. I have modified my health checks as: Protocol: HTTP, Path: /, Port: Traffic port, Healthy threshold: 5, Unhealthy threshold: 2, Timeout: 5, Interval: 30, Success codes: 200 but still, my targets health checks are failed with error: Unhealthy: health checks failed with these codes: [403] I don't know why it is giving error 403 even my instance ip is reachable. – marshmello Apr 14 '20 at 07:55
  • Update "Port" from "Traffic Port" to 80 and also make sure IP:Port is accessible. Or, you can host any static page, say index.html on apache, and update "Path" from "/" to "index.html" – saurabh14292 Apr 14 '20 at 08:42
  • I switched to from "Traffic Port" to 80 but still, it was showing Unhealthy: health checks failed with these codes: [403] . As maybe the problem is with the error code. When I ran curl -v http://'ip-address-of-instance':80/ it was giving 403 even web server was responding. So I set the success code from 200 to "200-499". is it OK? – marshmello Apr 14 '20 at 10:48
  • You can change the success code to mark the node healthy, however, in that case, codedeploy might work, but your application may not. Also, the issues now you are facing are not because of CodeDeploy. Kindly raise new questions for issues related EC2/HealthChecks. And if you find the answer I had posted for root cause of CodeDeploy failure, kindly accept the same. – saurabh14292 Apr 15 '20 at 02:39