0

I have a setup where I do batch computation using spot instances in an auto-scaling group.

The batch computation is coordinated by a "coordinator node", which is just an EC2 instance. I use an elastic ip to ensure the spot instances can connect to the coordinator node.

This has downsides: AWS limits the number of elastic IPs an instance can have & it seems unnecessary since I don't want to expose the coordinator to the outside world.

Is there a better way to have the instances in the auto-scaling group connect to the coordinator node?

Foobar
  • 115
  • 7
  • You can put in a support case for a limit increase to the number of Elastic IPs, but as mentioned in the answer, I'd just connect via private IPs. You also may want to consider using AWS Batch instead – Shahad Mar 01 '21 at 03:18

1 Answers1

0

The spot instances are inside the VPC so you don't need them to connect via the EIP of your coordinator. you can simply use the private address, which is also known and should be permanent.

Oscar De León
  • 241
  • 1
  • 2
  • 6
  • Why would the private IP address be permanent? I thought non-elastic IPs could change whenever an instance is turned on/off? – Foobar Mar 05 '21 at 02:52
  • You can either set it as fixed or _hope_ the DHCP will remember the address it had assigned. You're right that non-elastic public addresses will change when the instance is turned off and back on (I've noticed it doesn't happen in a clear reboot). – Oscar De León Mar 05 '21 at 08:12
  • What do you mean "set as fixed"? The reason I'm using a EIP is because it never changes -- if there's a way to have a fixed IP without using a EIP, I'd do that instead. – Foobar Mar 06 '21 at 01:11
  • You have the option of setting the private IP address to a fixed value when spinning up a new instance. – Oscar De León Mar 06 '21 at 09:06
  • Sorry. Hit the ENTER key too soon on that one... https://forums.aws.amazon.com/thread.jspa?threadID=222861 and, in my experience, it doesn't change when stopping/starting the instance. But if your instance is already up and running and you need to set the value of the IP address to be sure, the one way I've found is to add a new ENI and then do the change to make it primary at the OS level. – Oscar De León Mar 06 '21 at 09:14
  • 1
    Private IP addresses stick around when an instances is stopped. Its public IPs that go away when they stop (and a new one gets assigned when it starts). You need to use an EIP for a static /public/ IP. The reason for this is that private IPs are free (local to the VPC) but public IPs are expensive so AWS doesn't want your instance to have one assigned to it while not running – Shahad Mar 06 '21 at 18:14