43

I have a long list of machines, all of which are a little different in functionality in a system. I'd like to organize these machines and add to a hosts inventory file automatically so I can run ansible and manage inventory. Are there good solutions out there for this?

I think ansible hosts should looks something like...

[webservers]
someip
someip
[integration]
someip
someip

etc..

After asking the question, I currently am researching output vars and using those to render a template from a file.

DrM
  • 1,092
  • 1
  • 8
  • 11
  • 1
    can you add a bit of detail about how your terraform is organized and how your expect your inventory file to be organized? if you output machine IPs from terraform you can use the command `terraform output ` to return a list of IPs that can be a set of inventory for your playbooks. – RaGe Aug 03 '17 at 16:58
  • That is helpful. Actually I am thinking that perhaps I should use a combinations of templates and output variables to render. I found the following repo with code https://github.com/opencredo/k8s-terraform-ansible-sample/tree/master/terraform where the person renders an ssh configuration. – DrM Aug 03 '17 at 17:18
  • 1
    just so you know you can also run playbooks from inside terraform using a null_configuration resource. – RaGe Aug 03 '17 at 19:42
  • 1
    Are you aware of this: https://github.com/adammck/terraform-inventory? I found it some time ago by browsing Terraform tags on Github (cool feature by the way). – Matthew Schuchard Aug 03 '17 at 20:01
  • No, that is a pretty slick idea, converting it over with a terraform state file. I also appreciate the note from RaGe about running ansible using the null resource. It sounds like a nice trick, but how you would only run it if state is changed in that scenario - not sure. – DrM Aug 03 '17 at 20:53
  • You add a trigger to the null_resource that depends on the pertinent resource. an EC2 instance for example. Whenever the EC2 is recreated, the null_resource is triggered. This is sort of off-topic here, if you post a new question I will post a more detailed answer. – RaGe Aug 08 '17 at 14:31

7 Answers7

35

I figured it out.

data "template_file" "dev_hosts" {
  template = "${file("${path.module}/templates/dev_hosts.cfg")}"
  depends_on = [
    "aws_instance.dev-api-gateway",
    "aws_instance.dev-api-gateway-internal",
    ....
  ]
  vars {
    api_public = "${aws_instance.dev-api-gateway.private_ip}"
    api_internal = "${aws_instance.dev-api-gateway-internal.private_ip}"
  }
}

resource "null_resource" "dev-hosts" {
  triggers {
    template_rendered = "${data.template_file.dev_hosts.rendered}"
  }
  provisioner "local-exec" {
    command = "echo '${data.template_file.dev_hosts.rendered}' > dev_hosts"
  }
}

Then create a template in the file referenced earlier

Contents of example dev_hosts.cfg

[public]
${api_public}


[private]
${api_internal}
Andrei Sinitson
  • 739
  • 6
  • 10
DrM
  • 1,092
  • 1
  • 8
  • 11
  • 1
    Note that if you are using Terraform on Windows (unlucky you!) via something like Git Bash you also need to add `interpreter = ["sh", "-c"]`, otherwise it will use `cmd` and will fail to save the file. – Andrei Sinitson Nov 09 '18 at 13:31
  • 2
    Another hint: if you have a list (for example if you use `count`) and not a single IP, then you need to interpolate it to simple value - a string in the `vars` section. This is how to do it: `servers = "${join("\n", module.servers.server_ip)}"` – Andrei Sinitson Nov 09 '18 at 13:51
  • 1
    Expanding on that, if you have a list and want a complex expression, you can do something like this: `"${join("\n", [for instance in aws_instance.web-servers : join("", [instance.tags.Name, " ansible_host=", instance.public_ip])] )}"` – Datum Geek Aug 30 '19 at 11:55
26

Since Terraform 0.12+ there is templatefile function that comes handy, especially if you need to populate host groups:

# generate inventory file for Ansible
resource "local_file" "hosts_cfg" {
  content = templatefile("${path.module}/templates/hosts.tpl",
    {
      kafka_processors = aws_instance.kafka_processor.*.public_ip
      test_clients = aws_instance.test_client.*.public_ip
    }
  )
  filename = "../ansible/inventory/hosts.cfg"
}

Where hosts.tpl template file (future Ansible inventory) may look like this:

[kafka_broker_hosts]
%{ for ip in kafka_processors ~}
${ip}
%{ endfor ~}

[test_client_hosts]
%{ for ip in test_clients ~}
${ip}
%{ endfor ~}

End result:

[kafka_broker_hosts]
18.224.140.239
18.224.140.234

[test_client_hosts]
3.21.134.83
Andy Malakov
  • 797
  • 8
  • 6
  • Is there a way to add hostsnames with count using this approach,for example: [kafka_broker_hosts] kafka-1 18.224.140.239 kafka-2 18.224.140.234 [test_client_hosts] test-1 3.21.134.83 – Shahar Hamuzim Rajuan Jan 06 '22 at 12:50
  • if the resources are created using `for_each` option instead of count, content should be manipulated like this `workernode = values(aws_instance.kafka_processor)[*].public_ip` – Vignesh Muthu.S Dec 12 '22 at 08:02
19

Our approach is slightly different. We define a Terraform Module (terraform-null-ansible) which calls ansible anytime we want to run a playbook on a host by using a dynamic inventory.

https://github.com/cloudposse/terraform-null-ansible

This is a very terraform-centric approach but leads to very clean integration. Plus, by calculating the checksum of the playbook, we only call the ansible provisioner when the playbook has changed.

Usage is quite easy:

module "web_provisioner" {
   source    = "git::https://github.com/cloudposse/terraform-null-ansible.git?ref=tags/0.3.8"

   arguments = ["--user=ubuntu"]
   envs      = ["host=${aws_instance.web.public_ip}"]
   playbook  = "../ansible/playbooks/test.yml"
   dry_run   = false
}

More docs are on the GitHub README.md

Erik Osterman
  • 559
  • 4
  • 7
6

For multi severs:

File inventory.tf

data  "template_file" "k8s" {
    template = "${file("./templates/k8s.tpl")}"
    vars {
        k8s_master_name = "${join("\n", azurerm_virtual_machine.k8s-master.*.name)}"
    }
}

resource "local_file" "k8s_file" {
  content  = "${data.template_file.k8s.rendered}"
  filename = "./inventory/k8s-host"
}

File k8s.tpl

[kube-master]
${k8s_master_name}

final result

[kube-master]
k8s-master-01
k8s-master-02
k8s-master-03
Roy Zeng
  • 511
  • 7
  • 10
2

This worked for me on aws ec2:

main.tf:

resource "aws_instance" "instance" {
  for_each      = toset(["ingress-01", "node-01", "node-02", "master-01" ])
  ami           = "ami-0c239ecd40dcc174c"
  instance_type = "t2.micro"

  tags = {
    Name = "${each.key}"
  }
}

resource "local_file" "inventory" {
  content = templatefile("inventory.tmpl", { content = tomap({
    for instance in aws_instance.instance:
      instance.tags.Name => instance.public_dns
    })
  })
  filename = format("%s/%s", abspath(path.root), "inventory.yaml")
}

template (inventory.tmpl):

all:
  children:
    ingress:
      hosts:
%{ for content_key, content_value in content }
%{~ if length(regexall("ingress", content_key)) > 0 ~}
        ${content_key}:
          ansible_host: ${content_value}
%{ endif ~}
%{~ endfor ~}
    master:
      hosts:
%{ for content_key, content_value in content }
%{~ if length(regexall("master", content_key)) > 0 ~}
        ${content_key}:
          ansible_host: ${content_value}
%{ endif ~}
%{~ endfor ~}
    nodes:
      hosts:
%{ for content_key, content_value in content }
%{~ if length(regexall("node", content_key)) > 0 ~}
        ${content_key}:
          ansible_host: ${content_value}
%{ endif ~}
%{~ endfor ~}

cat inventory.yaml

all:
  children:
    ingress:
      hosts:
        ingress-01:
          ansible_host: ec2-xx-xx-xx-xx.eu-central-1.compute.amazonaws.com
    master:
      hosts:
        master-01:
          ansible_host: ec2-xx-xx-xx-xx.eu-central-1.compute.amazonaws.com
    nodes:
      hosts:
        node-01:
           ansible_host: ec2-xx-xx-xx-xx.eu-central-1.compute.amazonaws.com
        node-02:
           ansible_host: ec2-xx-xx-xx-xx.eu-central-1.compute.amazonaws.com
thdonatello
  • 21
  • 1
1

My approach: from template to inventory file, use template_file to render the content and use local_file to output a file.

template file:

## file inventory.tpl

[frontend]
${bastion_pub_ip}

[all:vars]
ansible_ssh_private_key_file = ${key_path}
ansible_ssh_user = ubuntu

render and output:

## file inventory.tf

data "template_file" "inventory" {
    template = "${file("./test/inventory.tpl")}"

    vars {
       bastion_pub_ip = "${element(azurerm_public_ip.bastion.*.ip_address, count.index)}"
       key_path = "~/.ssh/id_rsa"
    }
}

resource "local_file" "save_inventory" {
  content  = "${data.template_file.inventory.rendered}"
  filename = "./myhost"
}

It works for single server, if you have a list, I don't find a proper way to do it.

Roy Zeng
  • 511
  • 7
  • 10
0

This worked for me:

data "template_file" "ansible_inventory" {
  template = "${file("${path.module}/hosts.tmpl")}"

  vars = {
    public_ips = "${join("\n", aws_instance.public_instance.*.public_ip)}"
  }
}

resource "local_file" "hosts" {
  filename = "${path.module}/hosts"

  content = data.template_file.ansible_inventory.rendered
}
Marcelo Mourão
  • 11
  • 4