0

Can I send data via Infiniband without using a DMA-controller and what the smallest size of packages can I send?

That is, can I directly access to the memory of the remote CPU2-RAM from current CPU1-Core by using simple pointer (i.e. only x86-asm: MOV... sending data to PCI-Express BAR of Infiniband adapter) without DMA-controller?

An example, can I do something like this from CPU1:

unsigned char *rdma_ptr = get_rdma_pointer(CPU2);
rdma_ptr[3] = 111;
Alex
  • 12,578
  • 15
  • 99
  • 195

1 Answers1

2

In short - no. Longer version: if your aim is to write a simple code that can do it and not mess with all the RDMA things, you will need some middleware layer that will do all this and provide you with an easy abstraction. There are several such layers to use, such as UPC (e.g. Berkeley UPC implementation) or MPI (e.g. Open MPI) As for the package size - you can send no data that is payload), but if you do want the packet to be sent, there will be headers (I think it's 20 bytes or so, but it also depends on the protocol).

kliteyn
  • 1,917
  • 11
  • 24
  • Thanks! Besides MPI/UPC. I.e. to access by using simple pointer I must use some middleware layer that look like as memory mapping `mmap()` - which initiate page-fault on each access to this region and recieve/send data to/from remote server memory via Infiniband? However, does the intermediate layer is still use DMA-controller? – Alex Nov 20 '13 at 09:19
  • @Alex Any packet that is received by your network card (HCA) and carries data should result is something to be written somewhere in the host memory, and this memory is pre-allocated, pre-registered and pinned, and the data will be DMA'ed. – kliteyn Nov 20 '13 at 15:48