0

I am trying to install kypo cyber range by following its official guide in all-in-one mode of openstack deployment. While creating Base Infrastructure ./create-base.sh, I am getting below error exceeding maximum retries. It successfully built security security groups and networking stack. However head and proxy are generating Errors.

2022-03-30 07:53:00Z [kypo-head-stack]: CREATE_IN_PROGRESS  Stack CREATE started
2022-03-30 07:53:01Z [kypo-head-stack.kypo-head-port]: CREATE_IN_PROGRESS  state changed
2022-03-30 07:53:05Z [kypo-head-stack.kypo-head-port]: CREATE_COMPLETE  state changed
2022-03-30 07:53:08Z [kypo-head-stack.kypo-head]: CREATE_IN_PROGRESS  state changed
2022-03-30 07:53:08Z [kypo-head-stack.kypo-head-floating-ip]: CREATE_IN_PROGRESS  state changed
2022-03-30 07:53:25Z [kypo-head-stack.kypo-head-floating-ip]: CREATE_COMPLETE  state changed
2022-03-30 07:54:41Z [kypo-head-stack.kypo-head]: CREATE_FAILED  ResourceInError: resources.kypo-head: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 20fdda13-f5fa-4378-bf60-c64f0bf52371., Code: 500"
2022-03-30 07:54:41Z [kypo-head-stack.kypo-head]: DELETE_IN_PROGRESS  state changed
2022-03-30 07:54:46Z [kypo-head-stack.kypo-head]: DELETE_COMPLETE  state changed
2022-03-30 07:54:50Z [kypo-head-stack.kypo-head]: CREATE_IN_PROGRESS  state changed
2022-03-30 07:55:28Z [kypo-head-stack.kypo-head]: CREATE_FAILED  ResourceInError: resources.kypo-head: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 130604ae-a7d6-42f4-9010-56524561b4a0., Code: 500"
2022-03-30 07:55:28Z [kypo-head-stack.kypo-head]: DELETE_IN_PROGRESS  state changed
2022-03-30 07:55:35Z [kypo-head-stack.kypo-head]: DELETE_COMPLETE  state changed
2022-03-30 07:55:40Z [kypo-head-stack.kypo-head]: CREATE_IN_PROGRESS  state changed
2022-03-30 07:56:21Z [kypo-head-stack.kypo-head]: CREATE_FAILED  ResourceInError: resources.kypo-head: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad4be42c-fecb-43bc-881b-b8d6f0e17527., Code: 500"

Stack kypo-head-stack CREATE_FAILED 

2022-03-30 07:59:53Z [kypo-proxy-jump-stack]: CREATE_IN_PROGRESS  Stack CREATE started
2022-03-30 07:59:54Z [kypo-proxy-jump-stack.kypo-proxy-jump-port]: CREATE_IN_PROGRESS  state changed
2022-03-30 08:00:03Z [kypo-proxy-jump-stack.kypo-proxy-jump-port]: CREATE_COMPLETE  state changed
2022-03-30 08:00:03Z [kypo-proxy-jump-stack.kypo-proxy-jump-floating-ip]: CREATE_IN_PROGRESS  state changed
2022-03-30 08:00:05Z [kypo-proxy-jump-stack.kypo-proxy-jump]: CREATE_IN_PROGRESS  state changed
2022-03-30 08:00:15Z [kypo-proxy-jump-stack.kypo-proxy-jump-floating-ip]: CREATE_COMPLETE  state changed
2022-03-30 08:00:48Z [kypo-proxy-jump-stack.kypo-proxy-jump]: CREATE_FAILED  ResourceInError: resources.kypo-proxy-jump: Went to status ERROR due to "Message: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8d300d3f-1fdd-430e-b271-57eb5b1cbb8a., Code: 500"
...

Stack kypo-proxy-jump-stack CREATE_FAILED
aneela
  • 1,457
  • 3
  • 24
  • 45
  • Do you enough available resources to launch the stack? The error messages all refer to "instance ...", check the nova-compute.log as well as nova-scheduler.log and nova-conductor.log, also placement log could reveal any resource bottlenecks. – eblock Mar 30 '22 at 09:46
  • @eblock nova-compute.log says `: libvirt.libvirtError: unsupported configuration: Emulator '/usr/bin/qemu-system-x86_64' does not support virt type 'kvm'` I am using vmware with 4 VCPUs and 10G Ram – aneela Mar 31 '22 at 05:43
  • 1
    Then you probably should change the `virt_type` in your nova.conf to match your actual setup. – eblock Mar 31 '22 at 06:54
  • I did change it, but after running with new configuration, it is still generating previous error like changes are not being enabled. I am changing in nova-compute/nova.conf. – aneela Mar 31 '22 at 07:16
  • How exactly did you setup your all-in-one deployment? I'm not familiar with these tools like devstack etc. But maybe someone else can chime in then. Just to make sure, after changing nova.conf you restarted the service, correct? – eblock Mar 31 '22 at 07:42
  • I have used kolla ansible to setup deployment. Now I am building instances for kypo head and proxy. I have tried service openstack-nova-compute restart but it saying unit nova-compute.service not found. However logs are working. – aneela Mar 31 '22 at 09:47
  • So which nova process is running then? – eblock Mar 31 '22 at 10:29
  • I eventually restart the vm in order to get rid of all hustle. Now there is another error showing `Message: No valid host was found. There are not enough hosts available., Code: 500"`. When I look at nova-compute log it says `Failed to connect to libvirt: unable to connect to server at '192.168.211.133:16509': Connection refused'`. Internet says checks for baremetal nodes but I am running vmware .. – aneela Mar 31 '22 at 11:45
  • You should ensure that nested virtualization is enabled, I believe otherwise it won't work. – eblock Mar 31 '22 at 13:31
  • Nested Virtualization is enabled in vmware and kvm is also working fine now. I am still getting this error. – aneela Apr 01 '22 at 11:47

1 Answers1

0

Further log exploration of nova-compute.log said that libvirt.libvirtError: unsupported configuration: Emulator '/usr/bin/qemu-system-x86_64' does not support virt type 'kvm'. I changed the virt_type to qemu but error some how persisted. The root cause in my problem was I was running VM on VMWare without enabling Virtulize Intel VT-x/EPT which enables nested virtualization.

I check this box under VM Settings > Processor and the error went away.

aneela
  • 1,457
  • 3
  • 24
  • 45