Howto uniquely claim an element from a list in terraform?

Question

I'm looking for a way to uniquely assign an element from a list to a resource in terraform.

I find it hard to explain so I hope the following explains what I want to achieve:

`main.tf`

locals {
  networks = toset(["foo", "bar", "baz"])
  available_vlans = [
    100, 101, 102, 103, 104, 105, 106, 107,  108, 109,
    110, 111, 112, 113, 114, 115, 116, 117,  118, 119
  ]
}

# This is an example, This mimics something that creates a network
# in a device.
resource "local_file" "network" {
  for_each = local.networks
  filename = "${path.module}/${each.value}.mock"
  content = "???" # How do I claim a value from "available_vlans"
                  # which has not been taken already by another network?
}

So my question is, how can I tell terraform the following:

For each resource "localfile.network" pick a unique item from the list "available_vlans".
When a resource -let's say "bar"- is being removed, that value must be placed back in the list.
When a resource -let's say "bar"- is being removed, it must not change the vlan assigned to the resource "baz".

Would this be possible or is my train of thought a complete train wreck? :)

How would TF know what has been "taken already by another network"? Will you provide that as an input argument? — Marcin, Apr 22 '22 at 09:06
Hi Marcin, that is exactly what I am asking :) I could imagine there was a "pool" provider that i would populate in the provider config with a list of available values. And then a "available_item" resource that would claim such a value from that list. Then I could reference to that resource holding that value from other resources. — Goes S'epouse, Apr 25 '22 at 06:56

Oscar Drai · Answer 1 · 2022-04-22T15:00:54.140

0

For this kind of logic, I think you have two options:

Either just provide the network to vlan mapping yourself as an input so that you can simplify the logic.
Or use something such as SSM parameter to store the vlans being used and retrieve it with a data resource. You can then calculate the difference of the lists/sets and get the available vlans. If you assign them in order, it should be very easy to know the newly assigned vlans.

edited Apr 22 '22 at 15:00

answered Apr 22 '22 at 11:26

Oscar Drai

141
1
7

Hi Oscar, thank you for your answer. I was secretly hoping (not stated in my original question though) I could store these few bytes in the state db of terraform. To store it in a external system feels so.... overkill. edit: I explicitly do not want to supply the mapping myself, picking a number from a list is something a multicore something ghz machine should be more than capable of ;) – Goes S'epouse Apr 25 '22 at 06:51
You would need to use your own custom data source for that. Since its custom, you can program any logic you want, which would return you only valid and free networks. – Marcin Apr 25 '22 at 06:59

Martin Atkins · Answer 2 · 2022-04-22T18:23:04.783

Terraform is a "desired state" system, where configuration describes a set of objects that should exist and then Terraform itself (with the help of provider plugins) plans a set of changes to converge on that state.

Although it's possible to write configurations where the desired state is derived from existing objects in remote systems, it's important to ensure that the desired state you are describing won't change the source objects you were making the decision based on, because in a situation like that you can easily create a contradiction and therefore a configuration that can never converge because each time you apply it you change the desired state.

In your case, consider what would happen over the course of several runs if you start with none of the VLANs claimed:

On the first terraform apply, VLAN 100 is available and so will be used. Presumably a side-effect of applying that change is to "claim" that VLAN.
On the second terraform apply, VLAN 100 is no longer available and so this configuration would select VLAN 101. Applying this change would "claim" VLAN 101 as an updated VLAN for this resource, but would release VLAN 100.
On the third terraform apply, VLAN 100 is available again and so the configuration reverts back to claiming that one, releasing VLAN 101 back to the available pool again.
...

This cycle would continue indefinitely, because each time you apply a change you alter the source information that the desired state was based on and thus change the desired state.

To achieve a design like this will typically require something outside of Terraform to be issuing leases for VLANs and tracking when they are released so that they can be reused. An existing example of that in AWS is how you would typically declare an EC2 instance without specifying an explicit IP address for it and EC2 itself will choose an available IP address from the selected subnet and associate it with the EC2 instance's network interface.

The IP address is not specified as part of the desired state, so there is no contradiction: the instance just gets whichever IP address it gets, and the remote system keeps track of its assignment to that EC2 instance and returns it to the pool afterwards.

There may not be an existing similar ready-to-use system for assigning VLAN IDs as you want to do here, and so you may need to build that system yourself if you intend to manage these with Terraform. You might also conclude that Terraform is not a good tool to use for your particular problem, since you seem to need a system that is able to itself track leases of objects from a pool and Terraform is not designed to do that.

Hi Martin, thank you very much for your detailed answer. I'm aware how terraform functions and understand I cannot do this in terraform primitives (eg. list, object, etc). So a (simple?) provider for such functionality would also be of great use here. — Goes S'epouse, Apr 25 '22 at 06:44

Howto uniquely claim an element from a list in terraform?

main.tf

2 Answers2

`main.tf`