0

By creating an instance in OpenStack, IP is injected into the instance, but not other metadata as in Hostname and Key Pair. There's no error in the nova and neutron log files.

The problem:

...
checking http://169.254.169.254/2009-04-04/instance-id
failed 1/20: up 5.45. request failed
failed 2/20: up 7.47. request failed
...
failed 19/20: up 746.84. request failed
failed 20/20: up 795.86. request failed
failed to read iid from metadata. tried 20
failed to get instance-id of datasource
Top of dropbear init script
Starting dropbear sshd: failed to get instance-id of datasource
OK
...

The above snippet is part of the Cirros instance creation log. (The full log of the creation process is posted here)

The other error in the logs is this line: acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. Is this related to the issue?

Some of the config files:

# /etc/neutron/dhcp_agent.ini
[DEFAULT]
interface_driver = openvswitch
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = true
force_metadata = true


#/etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
type_drivers = vxlan,vlan,flat
tenant_network_types = vxlan
mechanism_drivers = openvswitch,l2population
extension_drivers = port_security
[ml2_type_flat]
flat_networks = provider
[ml2_type_vxlan]
vni_ranges = 1:1000


#/etc/neutron/plugins/ml2/openvswitch_agent.ini
[agent]
tunnel_types = vxlan
l2_population = True
[dhcp]
[network_log]
[ovs]
local_ip = 10.0.0.1
bridge_mappings = provider:br-provider
[securitygroup]
firewall_driver = iptables_hybrid

Compute node configurations: https://p.teknik.io/Ck1Ae

Here are errors from the cloud-init.log of the instance:

errors in cloud-init.log file

ip netns exec qdhcp-3b5bbc53-7638-4480-9349-407944933413 ip a

77: tap588fefad-6c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:df:33:e1 brd ff:ff:ff:ff:ff:ff
    inet 169.254.169.254/32 brd 169.254.169.254 scope global tap588fefad-6c
       valid_lft forever preferred_lft forever
    inet 10.0.1.1/24 brd 10.0.1.255 scope global tap588fefad-6c
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fedf:33e1/64 scope link
       valid_lft forever preferred_lft forever

Since an instance creation,

+ I have followed this instruction to implement self-service networking over OVS.

Moha
  • 3
  • 2
  • Have you also checked cloud-init.log? Maybe it shows why those other data have not been injected. You can test if the instance can query the metadata server with `curl http://169.254.169.254/latest` and then add the paths you want to check (like hostname etc.) – eblock Oct 28 '22 at 09:41
  • @eblock, From inside the instance, `curl` on http://169.254.169.254/latest waits and waits! I updated the post with errors from cloud-init.log: _Calling http://169.254.169.254/openstack failed ... timeout_ – Moha Oct 28 '22 at 19:40
  • Apparently the metadata service is not running or not responsive. Can you check neutron-metadata agent log and if the service is up? – eblock Oct 28 '22 at 23:05
  • I greped for ERROR/fail on the whole /var/log and there's nothing for these days! All services, included neutron-metadata-agent, are running. – Moha Oct 29 '22 at 07:51
  • Are any apparmor/selinux or firewall applications running? I’ve had similar effects with apparmor enabled. Has it ever worked for other instances or is this a fresh installation and the first attempt? – eblock Oct 29 '22 at 09:24
  • This is the first attempt over OVS; It's been working with LinuxBridge well; Now I'm trying to move it to OVS. Apparmor was running, I stopped it and created an instance, but the problem still exists: `checking http://169.254.169.254/2009-04-04/instance-id failed ` – Moha Oct 29 '22 at 13:52
  • The metadata agent should log something if the compute node tries to request metadata information for the instance, can you paste the relevant log output (not just grep for error)? And maybe also check (and paste) neutron-server.log output from the time of instance creation. And does the nova-compute.log reveal anything? Do you see the namespace for the magic ip (169.254.169.254) on the control node? I don't have access to an openstack env right now. – eblock Oct 30 '22 at 11:57
  • So each subnet with dhcp enabled should have a namespace containing the "magic ip", for example: `ip netns exec qdhcp-0db3acf6-d01e-484b-ace3-085422d90448 ip a` shows: `13: tapfe22e4cc-d9: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:26:27:76 brd ff:ff:ff:ff:ff:ff inet 169.254.169.254/32 brd 169.254.169.254 scope global tapfe22e4cc-d9` Do you see the same in your env for the subnet the instance has a tap device in? – eblock Oct 30 '22 at 14:28
  • Have you tried restarting neutron-metadata-agent? It should log something, at least the daemon restart. – eblock Oct 31 '22 at 19:08
  • "Empty since the beginning of instance creation"; restarting the service leads to log generation: http://p.teknik.io/Raw/oRcMe; Should I config something other than the Neutron itself and OVS on the computing side (http://p.teknik.io/Ck1Ae)? I have a controller node and a compute node. If I install and activate nova-compute on the controller node, instance creation works well, but still no metadata log; It just does not work if the compute node is separated! – Moha Nov 02 '22 at 07:42
  • Oh damn, I didn't think of it earlier. Are you using a provider network (you are according to `bridge_mappings = provider:br-provider`)? In that case try to start the instance with `config-drive = true`. – eblock Nov 02 '22 at 07:46
  • Either you enable the check mark if you create the instance via Horizon dashboard (Tab "configuration") or if you launch it via command line you add the option `--config-drive true` to the `openstack server create` command. Can you verify if config drive works? If it doesn't we might need to check the cloud.cfg within the vm image. – eblock Nov 02 '22 at 15:10

1 Answers1

0

One way to inject metadata into an instance is the config drive. If using the command line it can be added as an option to the openstack server create command:

openstack server create --flavor [...] --config-drive true my-vm

You can also force nova to always use a config drive, for that add this line to /etc/nova/nova.conf:

force_config_drive = true

Nova attaches a virtual drive to the instance where it stores the metadata, this drive is mounted by cloud-init and the desired config parameters are applied. It appears that in your case the metadata service does not work properly, I don't see a metadata_agent.ini section in your output, did you configure it? You can find more details in the installation guide.

eblock
  • 417
  • 2
  • 6