0

We are developing a CUDA-based system for a large statistical analysis.

I have a dual-socket motherboard, where each socket is assigned different PCI slots. 2x x16, 1 x8 for each LGA 2011 CPU (i.e. 40 lanes each). My question is: Can 1 CPU access GPUs on the other's PCI slots?

Obviously, since each each CPU only has 40 lanes, that's the max bandwidth available for data motion. And part of the program is multi-process, so we can utilize both CPUs and their GPUs.

But other parts are single process, and could make use of all available GPU resources (data contention aside) if available.

The other option is just buying a different MB with PLX support.

I'd test it, but we currently only have 1 CPU installed, and I'd rather use the $$$ to buy a different chassis and board(with PLX switching), if this won't work.

Thanks

Sheldon Ross
  • 5,364
  • 7
  • 31
  • 37

2 Answers2

0

Of course they can. Each CPU has a PCI-e controller for its PCI-e devices and that controller can be accessed by either CPU, just like all I/O devices.

David Schwartz
  • 179,497
  • 17
  • 214
  • 278
  • I assumed that, as something such as NIC you'd need available to both. I just wanted to be sure before dropping $2K on a second CPU only to not have it work out. – Sheldon Ross Aug 25 '15 at 18:10
0

Turns out currently Cuda can't communicate over QPI. So there while all the cards can be utilized, there is no peer-to-peer communication available between the cards on one processor, and the other.

So, while the answer is yes, it does have significant caveats.

Sheldon Ross
  • 5,364
  • 7
  • 31
  • 37