9

If I use my machine [Ubuntu 16.04 64 bit, kernel 4.4] for a while, QEMU needs the kernel caches to be dropped, otherwise, it won't manage to allocate the RAM.

Why does it happen?

This is a sample run:

~$ free -m
              total        used        free      shared  buff/cache   available
Mem:          15050        5427        3690          56        5931        4803
Swap:             0           0           0

~$ sudo qemu-system-x86_64 -m 10240 # and other options
qemu-system-x86_64: cannot set up guest memory 'pc.ram': Cannot allocate memory

~$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3

~$ free -m
              total        used        free      shared  buff/cache   available
Mem:          15050        1799        9446          56        3803        9414
Swap:             0           0           0

~$ sudo qemu-system-x86_64 -m 10240 # and other options
qemu-system-x86_64: cannot set up guest memory 'pc.ram': Cannot allocate memory

~$ echo 3 | sudo tee /proc/sys/vm/drop_caches
3

~$ free -m
              total        used        free      shared  buff/cache   available
Mem:          15050        1502       10819          56        2727       10784
Swap:             0           0           0

~$ sudo qemu-system-x86_64 -m 10240 # and other options
# Now QEMU starts
Peter Mortensen
  • 2,318
  • 5
  • 23
  • 24
Marcus
  • 242
  • 1
  • 3
  • 10

1 Answers1

19

Not all cached data can be discarded immediately. For example, cached dirty pages have to be written back to disk before they can be removed from RAM. You have no swap, so until those writes complete, there simply isn't enough available space for QEMU.

You really should add a reasonable amount of swap. You can't expect the memory manager to do a good job with one hand tied behind its back.

David Schwartz
  • 31,449
  • 2
  • 55
  • 84
  • 1
    As a theoretical question (since I'd like to learn more about how memory management actually works) why can't the manager delay (block) QEMU's memory allocation attempts while the dirty pages are being written back? – nanofarad Oct 09 '16 at 16:14
  • 2
    @hexafraction Just a guess: it's probably technically possible (but might add significant complexity, not sure), but the kernel devs would probably argue that there's no need for that feature, because the only problem it solves is caused by not having swap, which also causes other problems, all of which are fixed if you just enable swap and let the kernel do its memory management the way it's already coded to do well. – mtraceur Oct 09 '16 at 17:32
  • 1
    @hexafraction The kernel has no idea that's a sensible thing to do. For some applications, that makes no sense, so it's not the general policy. QEMU opted not to do that. – David Schwartz Oct 09 '16 at 17:37
  • 2
    @hexafraction Really, would you want to wait 30 seconds - or several minutes - for your `malloc()` call to _maybe_ find enough memory? – Michael Hampton Oct 09 '16 at 17:46
  • @MichaelHampton Fair enough. I think I understand the considerations now. – nanofarad Oct 09 '16 at 17:53
  • 3
    @hexafraction Think about it this way. If the kernel did theoretically have this feature to block a while if a malloc would otherwise fail, there would be no way to achieve the current behavior without additional APIs. On the other hand, the current implementation allows software that wants to wait and retry a while to retry malloc in a slow loop until it is satisfied. – Vality Oct 09 '16 at 19:30