qemu commandline can't see /dev/net/tun

Question

i am newbie with qemu and libvirt. I hit the issue with virsh and qemu, especially with tun interface setup. Once i run

qemu-system-x86_64 \
    -nographic -enable-kvm \
    -cpu host -m 2048 \
    -serial mon:stdio \
    -netdev tap,id=et0,ifname=tap0,script=no -device e1000,netdev=et0

It works without problem, it also creates tap0 interface at host machine. The problem comes up when i try to create VM using libvirt xml:

<domain type="kvm" xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>testttt</name>
  <uuid>21ddfabd-1312-45ad-aced-cf7a10257d83</uuid>
  <memory unit="MB">2048</memory>
  <currentMemory unit="MB">2048</currentMemory>
  <vcpu placement="static">1</vcpu>
  <os>
    <type arch="x86_64" machine="pc-i440fx-6.2">hvm</type>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on"/>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="vdi"/>
      <source file="..."/>
      <target dev="hda" bus="ide"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <controller type="usb" index="0" model="ich9-ehci1">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x7"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci1">
      <master startport="0"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0" multifunction="on"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci2">
      <master startport="2"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x1"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci3">
      <master startport="4"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pci-root"/>
    <controller type="ide" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
    </controller>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="vnc" port="-1" autoport="yes">
      <listen type="address"/>
    </graphics>
    <video>
      <model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/>
    </video>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/>
    </memballoon>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
    </rng>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-serial'/>
    <qemu:arg value='mon:stdio'/>

    <qemu:arg value='-device'/>
    <qemu:arg value='e1000,netdev=et0'/>
    <qemu:arg value='-netdev'/>
    <qemu:arg value='tap,id=et0,script=no,downscript=no,ifname=tap0'/>


  </qemu:commandline>
</domain>

The machine got created but once i try to run it i get folllowing error:

internal error: qemu unexpectedly closed the monitor:
2022-07-13T12:12:59.151035Z qemu-system-x86_64: -netdev tap,id=et0,script=no,downscript=no,ifname=tap0:
could not open /dev/net/tun: No such file or directory

I think, there is something wrong with:

  <qemu:commandline>
    <qemu:arg value='-serial'/>
    <qemu:arg value='mon:stdio'/>

    <qemu:arg value='-device'/>
    <qemu:arg value='e1000,netdev=et0'/>
    <qemu:arg value='-netdev'/>
    <qemu:arg value='tap,id=et0,script=no,downscript=no,ifname=tap0'/>


  </qemu:commandline>

I ran cat /dev/net/tun and get cat: /dev/net/tun: File descriptor in bad state so the tun should be ok, also it works in when i run qemu-system-x86_64.

score 2 · Accepted Answer · answered Jul 14 '22 at 15:40

Libvirt applies many layers of security protection around QEMU to prevent it accessing resources on the host that aren't explicitly assigned via the guest XML configuration. It can only automatically do the right thing for features configured using the officially supported XML elements. As soon as you use the <qemu:arg> override to setup arbitrary QEMU features, you're likely to get blocked by the security measures.

There's no especially strong reason to use the <qemu:arg> syntax in your case, since what you're doing is pretty simple.

   <serial type='pty'/>

is sufficient to setup a serial port, which you can interact with using 'virsh console $GUEST'.

Setting up 'mon' on stdio is not required, since libvirt talks to the QEMU monitor on behalf of the user. if there are edge cases where that's no good enough, you can talk to the QEMU monitor via virsh qemu-monitor-command

For the network interface, assuming you want to connect to a bridge

  <interface type='bridge'>
      <source bridge='br0'/>
      <model type='e1000'/>
  </interface>

though there are many other ways libvirt can setup the NIC.

https://libvirt.org/formatdomain.html#network-interfaces

If you absolutely must still use <qemu:arg> you'll need to disable various security features

https://libvirt.org/kbase/qemu-passthrough-security.html

score 1 · Answer 2 · edited Dec 16 '22 at 17:54

I had exactly the same problem as you: run with qemu is ok, but the error occurs with libvirt. The reason that i use qemu:commandline instead of libvirt network interface is libvirt will set virtio-net-pci backend default not virtio-net-device, but my guest kernel have no pci drivers.

Add device /dev/net/tun to cgroup_device_acl in /etc/libvirt/qemu.conf and restart libvirtd service could solve the problem perfect.

qemu commandline can't see /dev/net/tun

2 Answers2