1

I am using ECS Fargate model.I have two task defination and in each defination i have one service running (have place 1 container in each TD) Then i have two ECS services which run these two task definations. Network mode used is fargate.

I have created a load balancer which has two target groups.I have defined rules to divert traffic. In target groups i have added IP's of the services running inside two task definitions (as network mode is awsvpc i got ENI)

I have seen for one of the target group health check is failing continuously with HTTP code 502. I am observing the IP from ENI for that service changing continuously and same is updated in target group.

Questions:
Does ECS change IP in target group automatically ?
How to troubleshoot this HTTP code 502 as this is fargate i even cannot login inside container ?

Software Engineer
  • 15,457
  • 7
  • 74
  • 102
AWS_Lernar
  • 627
  • 2
  • 9
  • 26
  • 5XX mean service is not reachable or the health path does not exist, How you configure the health check? did the fargate service allow traffic? https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_run_task_fargate.html? did you application exist `/` path as this the default path. – Adiii Nov 30 '19 at 12:07
  • @Adiii yes application path exists.Health check - Protocol :HTTP,Port traffic:traffic port,Healthy Threshold:3,UnHealthy Threshold:3,Timeout:10,Interval:30.This was passing health check earlier. – AWS_Lernar Nov 30 '19 at 12:17
  • Did you enable clouds watch logs for service? If yes did you see any error in the cloud watche logs – Adiii Nov 30 '19 at 12:28
  • @Adiii Yes in cloudwatch logs its shows "java.net.ConnectException": Connection refused.Its spring boot application. – AWS_Lernar Nov 30 '19 at 12:30
  • So the application not running that is the main reason, first fix the application – Adiii Nov 30 '19 at 13:03
  • @Adiii When i contacted developers they are saying there is no issue in code.Its working in their local. – AWS_Lernar Nov 30 '19 at 13:13
  • 1
    Is the code trying to connect some external resources? like DB or something else that make it crash? is the application picking correct environment varialbe? if public access is required did your fargate task can communicate outside world? is there NAT? these are the multiple paths can give you some clue – Adiii Nov 30 '19 at 13:18
  • This probably won't help much, but the underlying problem is that you're using Fargate. If you'd used EKS instead, not only would it be easy to configure health-checks, you also would have had an answer on SO by now. That's because for every person who knows Fargate well enough to help you out, there are a hundred others who know K8S instead and you're much more likely to get help from a very large group of experts than a small group of guys who've read the docs. On SO, there are approx. 350 fargate questions, and around 22k K8S questions. – Software Engineer Nov 30 '19 at 17:41
  • @Engineer Dollery But the decision to use fargate or K8s is not my.Some standards define by shops have to follow :( – AWS_Lernar Nov 30 '19 at 18:09
  • @AWS_Lernar - like I said, it probably wouldn't help. The whole industry has moved to k8s, you should start hitting your seniors with medieval weapons and see if they pull out a gun, then you can explain the simile to them. – Software Engineer Nov 30 '19 at 18:19
  • @ Engineer Dollery Sure – AWS_Lernar Nov 30 '19 at 19:50
  • First of all, as @Adiii said, you need to check on the application. If you're seeing a java Exception, it means that it's not running for some reason. My bet is on an external service being wrongly configured in that image or being orphaned (like a local dependency such as Redis, DB, another app, etc). Furthermore a ConnectException is indeed a connection that cannot be made somehow. – marianogg9 Jan 27 '20 at 15:30

0 Answers0