0

Just in the past hour, our AWS CodeDeploy deployments have started hanging before even having a Start Time when looking at the Deployment Details page. The Status stays at In Progress indefinitely. We have not changed any of our deployment lifecycle details, so that leads me to believe that this is either some kind of CodeDeploy outage, or some kind of fluke that I'm not sure how to reset (Stopping the deployment and starting another ends up in the same place).

Has anyone else experienced this problem? Any ideas how to fix it?

Andrew
  • 1,890
  • 3
  • 16
  • 35

3 Answers3

2

Check the host agent on your instances. It's possible that it stopped running.

Jonathan Turpie
  • 1,343
  • 10
  • 16
  • The codedeploy-agent service was still running on the instance. I tried restarting it, but the problem hasn't gone away. I _was_ able to get one build to go through, and another to start before hanging, so it seems to be somewhat intermittent :/ – Andrew Dec 18 '15 at 20:19
  • You'll have to look at the host agent logs then to see what's failing. – Jonathan Turpie Dec 18 '15 at 20:34
  • It's filled with a bunch of `2015-12-17 00:49:40 INFO [codedeploy-agent(24008)]: [Aws::CodeDeployCommand::Client 200 60.814757 0 retries] poll_host_command(host_identifier:"...")` statements, and now I'm getting `Deployment d-94BFTBO0D hasn't completed adding instances. Deployment status: FAILED` errors from CodeDeploy. It seems like this is related to the degraded performance issue. – Andrew Dec 18 '15 at 21:30
  • Yea the new error is consistent with current event. Do you have logs from your previous deployments when you were getting the first error?. – Jonathan Turpie Dec 18 '15 at 21:46
  • It started working again right as they announced they had fixed it. Thanks for your help. – Andrew Dec 19 '15 at 22:49
  • For me, it was running on some and not on others, or it was frozen trying to start. This was on newly-spun-up instances, so no idea yet what's gone wrong. As soon as I started the agent on the servers, the codedeploy deployments succeeded. – David Aug 15 '17 at 07:05
1

it looks like there is currently degraded performance on the Virginia region EC2 API, I'm also having issues with CodeDeploy not working and I assumed it may be from the increase in error rates on DescribeInstances in that region. AWS status page

Charles Hearn
  • 26
  • 1
0

I had this happen and I kept restarting the pipeline to no avail. I finally pushed a minor change and it magically started working again.

Michael
  • 1,177
  • 15
  • 15