Run cloud-init cloud-config yaml file

Question

How do I run, for development purposes, cloud-init yaml file that will be normally run via user-data?

I know how I can re-run cloud-init, but I want to develop complicated cloud-init file and to do that it is rather difficult to continually build new instances.

Answer 1

Sorry to say, you're going to have to run it on a new clean instance (or at least a snapshot of one). Even if you did manually go back and start at different steps, there are potentially side effects.

I think you'll find that if you get used to managing local VMs, you can debug your scripts fairly quickly.

Answer 2

The quickest path for iterating on user-data input to cloud-init is probably via lxd. You can quickly set up lxd on a vm host or a bare metal system. Once set up, launches are very quick.

$ cat ud.yaml
#cloud-config
runcmd:
  - "read up idle < /proc/uptime; echo Up $up seconds | tee /run/runcmd.log"

$ lxc launch ubuntu-daily:bionic ud-test "--config=user.user-data=$(cat ud.yaml)"
Creating ud-test
Starting ud-test

$ lxc exec ud-test cat /run/runcmd.log
Up 8.05 seconds

$ lxc stop ud-test
$ lxc delete ud-test

Answer 3

You might be able to get away with just running cloud-init clean and then re-running it.

I'm experimenting with cloud-init and using an Ubuntu box with KVM as a virtualization lab. I made a simple Makefile to build the cloud-init image and launch it in an KVM instance.

You can see my code here:

https://github.com/brennancheung/playbooks/blob/master/cloud-init-lab/Makefile

all: clean build run

INSTANCE_NAME := "vm"
CLOUD_IMAGE_FILE = "bionic-server-cloudimg-amd64.img"
CLOUD_IMAGE_BASE_URL := "http://cloud-images.ubuntu.com/bionic/current"
CLOUD_IMAGE_URL := "$(CLOUD_IMAGE_BASE_URL)/$(CLOUD_IMAGE_FILE)"

download:
    wget $(CLOUD_IMAGE_URL)

clean:
    @echo "Removing build artifacts"
    -@rm -f config.img 2>/dev/null
    -@virsh destroy $(INSTANCE_NAME) 2>/dev/null || true
    -@virsh undefine $(INSTANCE_NAME) 2>/dev/null || true
    -@rm -f $(INSTANCE_NAME).img

build: 
    @echo "Building cloud config drive"
    cloud-localds config.img config.yaml
    cp $(CLOUD_IMAGE_FILE) $(INSTANCE_NAME).img

run:
    @echo "Spawning instance $(INSTANCE_NAME)"
    virt-install \
        --name $(INSTANCE_NAME) \
        --memory 8192 \
        --disk ./$(INSTANCE_NAME).img,device=disk,bus=virtio \
        --disk ./config.img,device=cdrom \
        --os-type linux \
        --os-variant ubuntu18.04 \
        --virt-type kvm \
        --graphics none \
        --network bridge=br0

Answer 4

I am not sure why this answer is not here.. maybe it is not applicable for earlier versions.

All I do to re-run cloud-init for dev testing is (, especially when testing user-data changes): 1 - change the config file/files, usually only /etc/cloud/cloud.cfg 2 - run clean:

cloud-init clean -l 

-l cleans the logs also 3 - re-run cloud-init

cloud-init init

of course, this has its limitations, depending on the settings you test, cloud-init clean is not going to revert the previous changes, but maybe you'll be able to figure out ways to test. For example I am testing the creation of new users, and every time I change something in the settings for a user and I want to test it.. I create a new user.

Yes, all this is quick in-development test, if you need to truly verify your changes - you need new instance.

Answer 5

Re-running all of cloud-init without system reboo isn't a recommended approach because some parts of cloud-init are run at systemd generator timeframe to detect new datasource types. That said, the following commands will allow you to accomplish this without reboot on a system.

cloud-init supports a clean subcommand to remove all semaphore files and allow cloud-init to re-run all config modules again. Beware that this will mean SSH host-keys are regenerated and .ssh config files re-written so it could impact your ability to get back into the VM.

To clean all semaphores so cloud-init modules will all re-run on next boot:

sudo cloud-init clean --logs

cloud-init typically runs multiple boot stages in sequence due to systemd service dependencies. If you want to repeat that process without a reboot you can run the following 4 commands:

1. Detect local datasource (cloud platform) and obtain user-data:

sudo cloud-init init --local

2. Detect any datasources and user-data which require network up and run cloud_init_modules defined in /etc/cloud/cloud.cfg:

sudo cloud-init init

3. Run all cloud_config_modules defined in /etc/cloud/cloud.cfg:

sudo cloud-init modules --mode=config

4. Run all cloud_final_modules defined in /etc/cloud/cloud.cfg:

sudo cloud-init modules --mode=final