2

Problem: Very slow IO in Windows Guest vs Host

Hardware: HP 360 G9, 2 x 10 cores Xeon CPU, 128GB Ram, 4 x 4TB SAS HDD, Raid 10, P440 Controller (with 2GB cache and battery)

Host: Ubuntu 20.04 Server, kvm virtualization, everything stock installed

Guest: Windows Server 2016, Windows Server 2019, Windows 10, any Windows basically, virtio drivers, at least 10 cores allocated, 50GB RAM allocated

We tried all combinations posible regarding cache and io values, tried multiple versions of virtio drivers, still no idea what we can do to improve performance.

Storage setup looks like this for the VM:

<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='unsafe' io='native'/>
  <source dev='/dev/sda4'/>
  <target dev='vda' bus='virtio'/>
  <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</disk>

The most impact is seen in manipulation of small files (4k - 16k). We have a folder with some 1GB of small files. On host the copy operation takes 10sec, on host about 1 min 30 sec!!!

There are tones of threads but could not find one that would help! Any ideas? I've read some complicated posted but still dont' know what I should try next!

  • setup scheduler on host?
  • configure the kernel on host to make sure it's using virtio modules?
  • change completly the host from Ubuntu to Fedora?

Thanks, Radu

Later Edit: Installed VirtualBox on host as hypervisor, there seems to be the same problem regarding the IO on the Windows VM's. The IO just does not perform.

Maybe it's a setting on the hardware, something that needs to be done in BIOS? Or something in the Host Operating System!?

Enovator
  • 31
  • 1
  • 6
  • I wouldn't use `cache='unsafe'` even with a battery backed hardware RAID. You cannot be sure that a guest write has ever actually been passed to the RAID controller as sync calls are ignored. You should try all of its other valid options to see if performance improves. My guess is `none` or `directsync` will do it. – Michael Hampton Mar 13 '21 at 22:26
  • Thanks Michael, tried that option too. No performance diffrence. The only thing that has some impact is stopping Windows Defender real time scanning in the VM. There seems that scanning small files has big impact on the VM. Still no clue on what's the problem and what can be done! – Enovator Mar 14 '21 at 09:32
  • Which option? I named two different ones you could try. – Michael Hampton Mar 14 '21 at 19:46
  • I've tried all of them, not just the 2 mentioned. On the VM IO remains painfully slow. A fraction of what the host can do. The strange thing is the on the same host, and other hypervisor (oracle virtual box) we have the same bad result. It's something not properly setuped on the hardware? On the Host OS? Should we ditch ubuntu and go to vmware or hyper-v? – Enovator Mar 14 '21 at 21:05
  • It could possibly be Ubuntu. All my KVM hypervisors are Red Hat and I have never seen such an issue. If you aren't in production yet, you definitely should experiment. – Michael Hampton Mar 14 '21 at 21:16
  • Copy operations in Windows Explorer are affected also by Windows caching. I'd advise to copy the same folder with xcopy (or robocopy) without buffering (switch /J) for test. – batistuta09 Mar 15 '21 at 13:13

0 Answers0