We've come to the stage of our project where we want to take the software to the next level, generally response times of 20µs or less. Software wise its as good as it gets (though there's room for a little more improvement), however now we're looking to attack the hardware side of things.
I was wondering if you could provide suggestions on how we can accomplish this feat. Do we need any special type of cabling, software stacks, architectures or other technologies?
Ultra low latencies? You get to play with the fun toys. The usual platform for this kind of really low latency networking would be InfiniBand (a couple of us, though not me, have gotten to use this stuff). Used in high-performance-computing and other computing areas that require low-latencies, high bandwidth, and scalability. Commonly found in certain scientific computing areas, as well.
Part of how IB gets the low latencies it does is by flattening the network. Similar to Fibre Channel, it is a layer 2 switched network which really helps bring down latency.
You won't find the gear to do this at your local MicroCenter. Using IB for networking is entirely doable, though TCP/IP is perhaps a mediocre choice of protocol if you want the lowest latencies possible.
Because of the low volume in the market, IB network adapters are very expensive compared to 1GB Ethernet, though they do compare favorably with 10GbE adapters. IB switches are similarly expensive, though seem to be near price-parity with 10GbE switches of similar port density.
If IB makes you leery, 10 Gigabit Ethernet can get you most of the way there and is very well supported by everything and is familiar to run of the mill network geeks. If you design your Ethernet so there is one, maybe 2, switches in it, and all ports are in the same subnet, you can get similar low latencies. Depending on exactly how much latency you're willing to tolerate, this may be a more understandable architecture than IB would be.
You are not even telling which network technology you are planning to use.
Assuming Ethernet and dividing the minimum frame length of 520 bytes by your 20 µs RT requirement gets you somewhere into the range of 50 MB/s. Further assuming that your software needs at least half of the time for processing, you are at 100 MB/s and probably looking at 10GE at that point to be able to accommodate for switching and transmission latency.
Then again, you do have plenty of headroom - so I am wondering why you are bothering with things like cabling types which surely will not affect your latency to the order of magnitude where it would get noticeable.
Edit: I'm used to believe that the solution depends on the particular problem. While 10GE latency and transport characteristics might be good for one application, they might present a problem for another. With 10 GE you would not have guaranteed RTTs but just best-effort delivery. This might be enough if your network is sufficiently over-engineered and it is okay for your application to get a response within 20 µs "most of the time". If the response time is a hard limit and a transaction would break because of this, you might want to look into other (mostly cell-based) networking technologies e.g. ATM.
As for Infiniband, you are probably looking into a major rewrite of your application's networking operations for the use of RDMA, if you want efficiency.
Another thing to consider is the distance between components - 10 µs is just enough time for light to travel 1,6 km and back - without any components latency and without considering bit times. So you would have to have short distances and short request/response sequences in order to get there.
All summed up it is not a question of individual components, but of the chosen technology.
