1

We have recently started bidding for spot instances on AWS. Since then prices in the availability zone we use have spiked at a price point suspiciously near our bids. I was trying to check this price with what other people see, and there are weird discrepancies.

I have compared EC2 prices for c1.medium on US-East:

prices taken from EC2

with those from the website EC2 Spot Prices:

enter image description here

But they do not match. What am I doing wrong? All parameters match; the unofficial site lacks us-east-1b, but the rest should be the same. Is it possible that Amazon changes prices depending on the customer? It would be interesting if someone else could confirm.

All the answers on ServerFault that I have checked, like this or this, always assume that there is a fixed price. This interesting article discusses the pricing algorithm but again there is no mention of different prices per customer.

Update: from a second AWS account I see different prices than from the first:

enter image description here

Even comparing zone per zone they are similar but not equal. They are also sort of compatible with those on the second site, but not equal. And for this second account zone us-east-1b is missing, something that also happens with other instance types. Curioser and curioser!

Community
  • 1
alexfernandez
  • 131
  • 7
  • AZ1 on the first account may be AZ3 on the second account and may be AZ2 on my account and may be AZ4 on my company's account. – Skaperen Sep 06 '16 at 06:00

2 Answers2

4

The paper you cited reflects observations and conclusions that are no longer applicable, if indeed they ever were.

Since then prices in the availability zone we use have spiked at a price point suspiciously near our bids.

This is actually not a surprise.

Spot capacity is allocated by the following logic, for each combination of availability zone, platform, OS type, and instance type:

  • Determine the number of instances available as spot instances of this type. (The algorithm here is unknown, but it is presumably some fraction of the available hardware for on-demand but not currently in-use.) Call this number n.

  • Sort the bids from high to low, and identify the top n largest price bids. These are the winning bids. If any non-winning bids have a running spot instance, terminate it. If any winning bids do not have a running instance, launch it. If winning bids alread have their running instance, do nothing.

  • Take the bid price of the lowest winning bid. This becomes the current price.

  • When bids are created or destroyed, or there is a shift in available capacity, start over from the top.

From this logic, we can extrapolate several conclusions:

  • If the spot price is at the minimum price, then there is more capacity available than there are bidders bidding over the absolute minimum.

  • Conversely, when the price is above the minimum, then demand exceeds capacity, so all bidders above the market price impact the market price upward, because the price is exactly the amount bid by the lowest winning bidder.

  • The closer the market price rises near your bid, the closer you are to being among the lowest winning bidders.

Contemplation of the algorithm suggests that if you are running multple instances in the same availability zone/platform/OS-type/instance-type, but you are willing to accept fewer than the desired number of instances, you should make multiple, individual bids with staggered bid amounts. This potentially gives you better insight into the market and increases the likelihood that you'll have some winners rather than none, and that if one of your bids is the lowest winner, the market price paid by all winners will be lower. If the price you have in mind is, say, $3/hr each for 3 instances, it might be desirable to instead bid $4, $3, and $2, because you would pay, maximum, $4 for 1, $6 for 2, and $6 for 3. If instead you bid $3 for 3, your maximum hourly cost would be $9 for 3 if your $3 bid is the lowest winner. The appropriate bidding strategy, however, depends on the value you place on the machines. I wrote at length about this on Webmasters.SE.

Note that, as you observed, within a region, across accounts, the availability zone-to-name mappings differ.

Q: How can I make sure that I am in the same Availability Zone as another developer?

We do not currently support the ability to coordinate launches into the same Availability Zone across AWS developer accounts. One Availability Zone name (for example, us-east-1a) in two AWS customer accounts may relate to different physical Availability Zones.

https://aws.amazon.com/ec2/faqs/#availability-zones

Community
  • 1
Michael - sqlbot
  • 22,658
  • 2
  • 63
  • 86
2

After looking carefully, prices are indeed the same for my two accounts, with a few caveats:

  • Regions are not consistent across customers. My 1d can be your 1a, and so on.
  • Some regions may be missing for some customers.
  • The site EC2 Spot Prices has some sampling issues that make its values less accurate and harder to see.
alexfernandez
  • 131
  • 7