How to copy file with variable changed to value from local to remote using Terraform?

Question

I am using Terraform to create a debian-cloud/debian-9 image on Google Cloud Platform. I have a directory on local that I am copying to the created instance using gcloud compute scp --recurse [LOCAL_DIR] [INSTANCE-NAME]:[REMOTE_LOCATION].
The directory on local has multiple .conf files which look something like below:

<source>
  @type tail

  format syslog

  path /var/log/syslog
  pos_file /var/lib/google-fluentd/pos/syslog.pos
  read_from_head true

  #Here's the variable I wanna replace
  tag ${instance-name}-syslog
</source>

I've created these .conf files and added the variable ${instance-name}. Now I would like that variable to be replaced by the value of Terraform/linux environment variable.

for e.g: if a Terraform/linux environment variable has a value like "some-value", then ${instance-name} from all the .conf files should be replaced by this and look something like below:

<source>
  @type tail

  format syslog

  path /var/log/syslog
  pos_file /var/lib/google-fluentd/pos/syslog.pos
  read_from_head true

  #Here's the variable I wanna replace
  tag some-value-syslog
</source>

I am looking to replace the value only on the remote (GCE instance) where I copied the directory, not in the files on my local.
Modifying the files to replace the variable on the server is also an acceptable option in my case, but I am not really sure if that is a good way to do this. If it is, I am not sure what script would read the files one by one and replace the variables.

EDIT : Adding the Terraform script to create a Debian instance and copy directory from local to server

resource "google_compute_instance" "default" {
  name         = "${var.instance_name}"
  project      = "${var.project}"
  machine_type = "${var.machine-type}"
  zone         = "${var.instance-zone}"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
    }
  }

  network_interface {
    network = "default"

    access_config {
      // Ephemeral IP
    }
  }


  #If replace the variables using shell script, this script will be used
  metadata_startup_script = "replace_var.sh"

  #One way
  provisioner "local-exec" {
    command = "gcloud compute scp --recurse [LOCAL-DIR] ${var.instance_name}:/etc/google-fluentd"
  }

  #Another way
  provisioner "file" {
    source      = "[LOCAL-DIR]"
    destination = "/etc/google-fluentd"
  }
}

Can you show the Terraform code that creates the image/instance including copying the config over? — ydaetskcoR, Apr 04 '19 at 21:47
@ydaetskcoR I have edited the question and added the Terraform script. — Amit Yadav, Apr 05 '19 at 05:34

score 2 · Accepted Answer · edited Apr 05 '19 at 16:01

You can use template_dir terraform resource to render the templates to a local directory and then upload them using file provisioner:

resource "template_dir" "config" {
  source_dir      = "${path.module}/path/to/fluent/templates/"
  destination_dir = "/tmp/fluent-templates"

  vars = {
    instance-name = "${var.instance_name}"
  }
}

resource "google_compute_instance" "default" {
  name         = "${var.instance_name}"
  project      = "${var.project}"
  machine_type = "${var.machine-type}"
  zone         = "${var.instance-zone}"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
    }
  }

  network_interface {
    network = "default"

    access_config {
      // Ephemeral IP
    }
  }

  provisioner "file" {
    source      = "${template_dir.config.destination_dir}"
    destination = "/etc/google-fluentd"
  }
}

You can also add a step to clear the temporary files generated:

  resource "null_resource" "cleanup" {
    depends_on = ["google_compute_instance.default"]

    provisioner "local-exec" {
      command = "rm -rf ${template_dir.config.destination_dir}"
    }
  }

Hope that helps.

This works well for generating the conf files from the templates. However, while trying to copy I get this error `* google_compute_instance.default: timeout - last error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain ` — Amit Yadav, Apr 05 '19 at 10:24
You need to add the ssh keys to your instance. Check this: https://stackoverflow.com/a/38647811/744997 — mukesh, Apr 05 '19 at 12:15

How to copy file with variable changed to value from local to remote using Terraform?

1 Answers1