We have a ceph cluster of four nodes, 48GB of memory on each, and Debian 9.8. Cluster connected 10Gbe. Cisco sg350xg-24t switch, Intel Corporation Ethernet Controller 10G X550T Cards. One port for internal network, one for external. For tests, each node has only one ssd Intel DC S4600 Series and BlueStor. Created rbd pool, located on these ssd with replication 3. Created image in this pool. The image is mounted on one of the nodes.
With the fio test, I get 600 IOPS on random write and 3,600 IOPS on random read. With the rbd bench test, I got 6000 IOPS on random write and 15000 IOPS on random read.
Can you please tell me where such a big difference in performance comes from?
There are also nodes with proxmox, when using Ceph Rbd, for virtual machines, there is the same performance as with fio.
