0

I've been having a very confusing discussion with a colleague about the use of Network Direct in Windows to provide RDMA, and Linux where we have a working implementation using the verbs api and rdmacm.

Said collegue developed his Network Direct implementation after he couldn't get the verbs one to work. I did the rdmacm implementation under Linux. We both got them talking linux=>linux and windows=>windows but never got around to testing Linux=>windows or windows=>linux. During the development there was some hint of it possibly working, they appeared to start talking but of course things were still in development.

The project was put on hold for months and the framework that it was built into has had significant redesign. I'm left with pulling what was working from the Windows side into the new framework and I'm having a hard time getting it working or seeing any part of it connect from Linux.

My understanding is that at the lower layer they will both talk verbs. But I'm now not so sure with colleague saying things like "Remember this is not verbs", just to add to the confusion.

So the question is. will an application that uses rdmacm be able to establish a connection to an app that uses Windows NetworkDirect interface or are the two completely different and incompatible?

hookenz
  • 36,432
  • 45
  • 177
  • 286
  • This probably depends on the RDMA/ND provider. – haggai_e Jul 23 '14 at 08:32
  • Well both from mellanox. – hookenz Jul 23 '14 at 19:53
  • According to this [thread](http://lists.openfabrics.org/pipermail/ofw/2014-January/008562.html) it should work. – haggai_e Jul 24 '14 at 04:36
  • Should... yeah it's not for me. works from Linux to Linux and Windows to Windows but not Windows to Linux. The thread you posted indicates if it doesn't work it's a driver bug. And the drivers I have are rather old because the cards I'm developing with are old. I'll have to try it out on the ConnectX cards we have in production. – hookenz Aug 01 '14 at 00:00

1 Answers1

1

According to Mellanox support, Network Direct should be compatible with InfiniBand verbs. We've had good luck doing RDMA transfers using rdmacm Linux-Linux and Windows-Linux. We used the windows libraries available at open fabrics alliance (look for ofed_sdk). It was not easy to figure out what libraries to get in order to do the Windows part.

However, Microsoft does not want people to use the ibverbs approach on MS-Windows any more. If you are having trouble getting ND on Windows to talk to Linux, are you sure that you are putting the values in network order before shipping them over?

user3885663
  • 26
  • 1
  • So the problem was that there was a driver bug which is why it wasn't working. Not unexpected since my dev hardware is much older DDR based compared to the prod hardware which is FDR. – hookenz Sep 15 '14 at 22:05