0

The CEPHFS cluster is deployed in vmware virtual machines,The virtual machine memory configuration is 4G Test with FIO

The order is as follows: fio -name=task1 -filename=/mnt/testcephfs/fio-test-1 -rw=randwrite -ioengine=sync -size=1G -bs=4M -iodepth=1 -direct=0 -numjobs=1 Please look at the picture 1 enter image description here

Then I adjusted the virtual machine memory to 8G The order is as follows: fio -name=task1 -filename=/mnt/testcephfs/fio-test-1 -rw=randwrite -ioengine=sync -size=1G -bs=4M -iodepth=1 -direct=0 -numjobs=1 Please look at the picture 2 enter image description here

The question is: Why do I increase the VIRTUAL machine memory and execute the same fiO command and bW increase?

fuzhiyu
  • 3
  • 3

1 Answers1

0

(This doesn't look like a programming question. Maybe it would have been better asked on https://unix.stackexchange.com/ or https://serverfault.com/ ? I also wonder if some of your dashes have been eaten - could you check the page about formatting posts over on https://stackoverflow.com/editing-help ?)

Your fio job means I/O is allowed to be staged in the page cache and control returned to fio for the next I/O even though it hasn't yet been sent to the backend. This decoupling often helps performance (e.g. because batching is improved) but it means care must be taken during benchmarking (because it's unclear what you've benchmarked).

If the page cache is "full" then control can't be returned to fio until there is space for the next I/O to be accepted. Thresholds as to when flushing of staged I/O starts are often related to fullness (but that's not the only criteria). Therefore:

  • If you are sending I/O fast enough the page cache might be filling up and fio can be forced to wait while it becomes free
  • If the page cache is allowed to bigger (e.g. because you have doubled the amount of memory the machine has) then maybe more I/O can be buffered and flushing may be allowed to start later

TLDR; you're seeing interference from caching and because you're doing so little I/O it could be massively distorting your numbers. Have you considered setting end_fsync=1 or doing dramatically more I/O than your memory?

Anon
  • 6,306
  • 2
  • 38
  • 56