4

Not sure if this is supposed to be on SuperUser or here, but here goes:

Q. If my spot instance is automatically terminated due to my max bid being exceeded, is it a clean shutdown -- as if a user clicked Start>>Shutdown ? Or is it a hard shutdown -- as if someone just yanked out the power cord?

I realize that in most cases the volumes on a terminated spot instance are deleted immediately, making my question moot. But in this particular case, I have the "delete volume on terminate" flag set to false so as to retain the root volume; I may want to reuse that volume and bind it to another instance in the future. Thus my concern regarding whether or not the forced shutdown is "clean." A clean shutdown would reduce or eliminate the possibility of data corruption on the root volume.

UPDATE (08/11/2013): I already marked a response as the answer but now I'm not sure it's correct. I just received a contradictory response from someone on the Amazon forum:

The instance will be shutdown gracefully and then the volumes with "Delete on Termination" set as false will be detached.

So, I'm still uncertain what the actual answer is. Anyone want to go for a tiebreaker on this one?

Chad Decker
  • 355
  • 1
  • 7
  • 12
  • 1
    Wait for your EBS-backed spot instance to be terminated, and then check its logs. If it was shut down gracefully, this will be quite obvious in the system log. – Michael Hampton Aug 11 '13 at 19:39
  • 1
    There are a few items that suggest an attempt is made to shutdown cleanly: from the AWS forums: ["we try to cleanly terminate"](https://forums.aws.amazon.com/thread.jspa?threadID=108815), ["should go through normal termination"](https://forums.aws.amazon.com/thread.jspa?threadID=40005), ["will be shutdown gracefully"](https://forums.aws.amazon.com/thread.jspa?messageID=477389); and from a non-amazon article: ["Amazon is nice enough to send a shutdown command on termination"](http://cloud.dzone.com/articles/hot-notifyemail-yourself-when). – cyberx86 Aug 11 '13 at 19:46

2 Answers2

7

Q. If my spot instance is automatically terminated due to my max bid being exceeded, is it a clean shutdown -- as if a user clicked Start>>Shutdown ? Or is it a hard shutdown -- as if someone just yanked out the power cord?

It might be a clean shutdown, but it also might be a hard shutdown. Amazon's wording implies they try to make it clean but don't make any promises.

Check this out: http://aws.amazon.com/ec2/spot-instances/#6

The highlights are this:

Because Spot Instances can be terminated without warning, it is important to build your applications in a way that allows you to make progress even if your application is interrupted.

And more importantly, this:

While we attempt to cleanly terminate your instances, your application should be prepared to deal with an immediate shutdown.

Drew Khoury
  • 4,637
  • 8
  • 27
  • 28
  • 1
    +1, it is my guess/understanding that EC2 will try the equivalent of "xm shutdown" and then, after some maximum waiting time, do an "xm destroy" if the instance is still around. – Luke404 Aug 28 '13 at 14:03
0

When max bid is exceeded, it goes down hard.

You will likely need to run fsck or chkdisk on the vols before mounting them.

EEAA
  • 109,363
  • 18
  • 175
  • 245
  • 1
    Which means also that database corruption would be a major issue. – Fiasco Labs Aug 10 '13 at 18:40
  • Not the answer I was hoping for but thanks. Good to know. – Chad Decker Aug 10 '13 at 19:12
  • 3
    Do you have a reference for this answer? – Eric Hammond Aug 10 '13 at 19:13
  • 1
    @EricHammond Amazon implies this in there wording, the best I could find is the following. http://aws.amazon.com/ec2/spot-instances/#6 Because Spot Instances can be terminated without warning, it is important to build your applications in a way that allows you to make progress even if your application is interrupted. While we attempt to cleanly terminate your instances, your application should be prepared to deal with an immediate shutdown. – Drew Khoury Aug 15 '13 at 06:51
  • 1
    @DrewKhoury Thanks for looking this up. That answer better matches my understanding of how it works. – Eric Hammond Aug 15 '13 at 21:30
  • @DrewKhoury's answer is correct - if the OP sees this, hopefully he will un-accept my answer and accept Drew's. – EEAA Aug 16 '13 at 21:25
  • 1
    @EEAA: You can edit you answer and point folks at the correct information. – Eric Hammond Aug 17 '13 at 19:17