2

Using cloud-init, how can I run add text (list of strings) to a file? Also, do I need to escape the colon : character? It seems that that problem is YAML validation but could not find any examples to help me out.

Here is what I've tried. None of the echo commands seem to be valid.

#cloud-config

runcmd:
  - aws s3 cp s3://my-bucket/elasticsearch/elasticsearch.tar.gz /opt/elastic/elasticsearch.tar.gz
  - tar -xf /opt/elastic/elasticsearch.tar.gz
  - ln -sv /opt/elastic/elasticsearch-7.7.6 /opt/elastic/elasticsearch
  - cp /opt/elastic/elasticsearch/config/elasticsearch.yml /opt/elastic/elasticsearch/config/elasticsearch.yml.bkp
  - echo 'cluster.name: DEMO' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - echo 'node.name: node1' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - echo 'path.data: /opt/elastic/data' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - echo 'path.logs: /opt/elastic/logs' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - echo 'network.host: host1.domain.internal' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - echo 'http.port: 9200' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - echo 'discovery.seed_hosts: ["host1.domain.internal", "host2.domain.internal", "host3.domain.internal"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - echo 'cluster.initial_master_nodes: ["node1", "node2", "node3"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
Cyrus
  • 84,225
  • 14
  • 89
  • 153
Morariu
  • 334
  • 2
  • 18

1 Answers1

6

You're insufficiently quoting the values in your list, so the : in your echo statements is getting interpreted as the key: value separator. You want to quote the entire contents of each line, and the best way of doing this is probably using one of the YAML quote operators (>, the folding quote operator, or | the literal quote operator). You'll find some documentation on this topic here.

Like this:

#cloud-config

runcmd:
  - |
    aws s3 cp s3://my-bucket/elasticsearch/elasticsearch.tar.gz /opt/elastic/elasticsearch.tar.gz
  - |
    tar -xf /opt/elastic/elasticsearch.tar.gz
  - |
    ln -sv /opt/elastic/elasticsearch-7.7.6 /opt/elastic/elasticsearch
  - |
    cp /opt/elastic/elasticsearch/config/elasticsearch.yml /opt/elastic/elasticsearch/config/elasticsearch.yml.bkp
  - |
    echo 'cluster.name: DEMO' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - |
    echo 'node.name: node1' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - |
    echo 'path.data: /opt/elastic/data' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - |
    echo 'path.logs: /opt/elastic/logs' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - |
    echo 'network.host: host1.domain.internal' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - |
    echo 'http.port: 9200' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - |
    echo 'discovery.seed_hosts: ["host1.domain.internal", "host2.domain.internal", "host3.domain.internal"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
  - |
    echo 'cluster.initial_master_nodes: ["node1", "node2", "node3"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml

You can verify this parses correctly by feeding it through a YAML-to-JSON converter, which will show you:

[
  "aws s3 cp s3://my-bucket/elasticsearch/elasticsearch.tar.gz /opt/elastic/elasticsearch.tar.gz\n",
  "tar -xf /opt/elastic/elasticsearch.tar.gz\n",
  "ln -sv /opt/elastic/elasticsearch-7.7.6 /opt/elastic/elasticsearch\n",
  "cp /opt/elastic/elasticsearch/config/elasticsearch.yml /opt/elastic/elasticsearch/config/elasticsearch.yml.bkp\n",
  "echo 'cluster.name: DEMO' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n",
  "echo 'node.name: node1' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n",
  "echo 'path.data: /opt/elastic/data' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n",
  "echo 'path.logs: /opt/elastic/logs' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n",
  "echo 'network.host: host1.domain.internal' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n",
  "echo 'http.port: 9200' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n",
  "echo 'discovery.seed_hosts: [\"host1.domain.internal\", \"host2.domain.internal\", \"host3.domain.internal\"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n",
  "echo 'cluster.initial_master_nodes: [\"node1\", \"node2\", \"node3\"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml\n"
]

You should be able to combine those lines into a single multi-line shell script, like this (I've taken the liberty of replacing the multiple echo statements with a single "here"-document):

#cloud-config

runcmd:
  - |
    aws s3 cp s3://my-bucket/elasticsearch/elasticsearch.tar.gz /opt/elastic/elasticsearch.tar.gz
    tar -xf /opt/elastic/elasticsearch.tar.gz
    ln -sv /opt/elastic/elasticsearch-7.7.6 /opt/elastic/elasticsearch
    cp /opt/elastic/elasticsearch/config/elasticsearch.yml /opt/elastic/elasticsearch/config/elasticsearch.yml.bkp

    cat >> /opt/elastic/elasticsearch/config/elasticsearch.yml <<EOF
    cluster.name: DEMO' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    node.name: node1' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    path.data: /opt/elastic/data' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    path.logs: /opt/elastic/logs' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    network.host: host1.domain.internal' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    http.port: 9200' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    discovery.seed_hosts: ["host1.domain.internal", "host2.domain.internal", "host3.domain.internal"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    cluster.initial_master_nodes: ["node1", "node2", "node3"]' >> /opt/elastic/elasticsearch/config/elasticsearch.yml
    EOF

That's probably both easier to read and easier to maintain.

larsks
  • 277,717
  • 41
  • 399
  • 399