terraform wants to replace ec2 instances when i simply want to add a rule to a security group

Question

I have an ec2 instance defined in terraform along with some security rules.

These are the security rules:

resource "aws_security_group" "ec2_web" {
  name        = "${var.project_name}_${var.env}_ec2_web"
  description = "ec2 instances that serve to the load balancer"
  vpc_id      = aws_vpc.main.id
}
resource "aws_security_group_rule" "ec2_web_http" {
  type              = "egress"
  from_port         = 80
  to_port           = 80
  protocol          = "tcp"
  # cidr_blocks       = ["0.0.0.0/0"]
  security_group_id = aws_security_group.ec2_web.id
  source_security_group_id = aws_security_group.elb.id
}
resource "aws_security_group_rule" "ec2_web_ssh" {
  type              = "ingress"
  from_port         = 22
  to_port           = 22
  protocol          = "tcp"
  cidr_blocks       = ["${var.ip_address}/32"]
  security_group_id = aws_security_group.ec2_web.id
}

I'm trying to simply add another security rule:

resource "aws_security_group_rule" "ec2_web_ssh_test" {
  type              = "ingress"
  from_port         = 22
  to_port           = 22
  protocol          = "tcp"
  cidr_blocks       = ["${var.ip_address}/32"]
  security_group_id = aws_security_group.ec2_web.id
}

And terraform wants to completely replace the security group, and that cascades into completely replacing the ec2 instance.

I'm modifying the .tf file and then running:

terraform apply

EDIT:

The security group itself seems completely unrelated. When I do "plan", I get the output:

  # aws_instance.ec2 must be replaced
-/+ resource "aws_instance" "ec2" {
...
      ~ security_groups                      = [ # forces replacement
          + "sg-0befd5d21eee052ad",
        ]

The ec2 instance is created with:

resource "aws_instance" "ec2" {
  
  ami           = "ami-0b5eea76982371e91"
  instance_type = "t3.small"
  key_name      = "${var.project_name}"

  depends_on = [aws_internet_gateway.main]

  user_data     = <<EOF
#!/bin/bash
sudo amazon-linux-extras install -y php8.1 mariadb10.5
sudo yum install -y httpd mariadb php8.1 php8.1-cli
sudo systemctl start httpd
sudo systemctl enable httpd
echo 'yup' | sudo tee /var/www/html/index.html
echo '<?php echo phpinfo();' | sudo tee /var/www/html/phpinfo.php
  EOF
  tags = {
    Name = "${var.project_name}_${var.env}_ec2"
  }
  root_block_device {
    volume_size = 8 # GB
    volume_type = "gp3"
  }
  security_groups = [aws_security_group.ec2_web.id]
  # vpc_security_group_ids = [aws_security_group.main.id]
  subnet_id = aws_subnet.main1.id
}

If I comment out

# security_groups = [aws_security_group.bake.name]

I do not get any errors.

@marcin, terraform plan gives the same output - plans to destroy the ec2 instances — iateadonut, Jan 11 '23 at 07:52
@marcin, i added the relevant info from plan and how the ec2 instance is created, as well as some other info. thanks. — iateadonut, Jan 11 '23 at 08:47

score 4 · Accepted Answer · answered Jan 11 '23 at 08:54

This happens because security_groups can only be used EC2-Classic (legacy instances) and a default VPC. For everything else you must use vpc_security_group_ids.

In your cause you are using custom VPC called main, thus you must be using vpc_security_group_ids, not security_groups.

score 0 · Answer 2 · answered Jan 11 '23 at 07:58

Before apply please run terraform apply -refresh-only then your main problem is you can not define same rule with different terraform id.

When you apply new changes for ec2_web_ssh_test AWS will complain about

│Error: [WARN] A duplicate Security Group rule was found on (sg-xxxxxx). This may be
│ a side effect of a now-fixed Terraform issue causing two security groups with
│ identical attributes but different source_security_group_ids to overwrite each
│ other in the state

Then you will get this error from AWS api

Error: InvalidPermission.Duplicate: the specified rule "peer: xxx.xxx.xxx.xxx/32, TCP, from port: 22, to port: 22, ALLOW" already exists

score 0 · Answer 3 · answered Jan 11 '23 at 08:36

With Terraform it compares the current state of your configuration with the new state which will contain the new rule you are adding. Here current state is not same as the desired state with new rule you are adding.

Hence with two different states, Terraform is trying to destroy the EC2 instances and trying to build new instances with newly added rules state.

This can be avoided with using terraform import command that will import the existing resources to your terraform state and then make changes to it.

terraform wants to replace ec2 instances when i simply want to add a rule to a security group

3 Answers3