1

I heard about that RDMA reads and writes are implemented like, when requests come the nic find the physical page and then using DMA to get the data to nic and then to the target.

This is straightforward for reads and write, but it is odd when it comes to the atomic operations. My question is that is RDMA atomic operations implemented the same way as reads and writes and how? And more specificely, what is it's relationship between the cpu's atomic operations (like compare and swap) and the RDMA's corresponding operations?.

haggai_e
  • 4,689
  • 1
  • 24
  • 37
user3008895
  • 11
  • 2

1 Answers1

4

RDMA atomic operations are implemented using PCI-express read and write operations. As such they do not provide atomicity with respect to the CPU's atomic operations, nor with respect to other HCAs.

haggai_e
  • 4,689
  • 1
  • 24
  • 37
  • Are there HCAs (say, Mellanox's ConnectX-4) that use PCIe atomics instead of read + write over PCIe? – Anuj Kalia Aug 25 '15 at 07:15
  • 2
    Current Mellanox HCAs do not support that feature. – haggai_e Aug 31 '15 at 07:06
  • Haggai, what about Connect X-5? http://www.mellanox.com/related-docs/user_manuals/ConnectX-5_VPI_IC.pdf "ConnectX-5 delivers ... additional Network Atomic and PCIe Atomic operations support. – PCIe Gen 4 ... – PCIe Atomic". Are host platform required to provide PCIe Atomic to use X-5? – osgx Aug 12 '16 at 03:42
  • 1
    Yes, the host CPU must support PCIe atomics. I believe that the PCIe controller on Ivy Bridge and newer Intel server CPUs supports PCIe atomics. Sandy Bridge CPUs support PCIe 3.0, but do not support PCIe atomics AFAIK. – Anuj Kalia Aug 24 '16 at 19:50