Is it possible to cross connect between different data centers? I'm looking to connect to a financial exchange and need extremely low latency. The cost for a data center 100 metres down the road is substantially less.
Is there a cost-effective way to have low latency between the 2 centers? What is the main cause of latency? The distance is negligble so I assume it must be firewalls or switches?
I would speak the the Data Centre Operations team at one of the DC's to find out if there is existing connectivity between the two sites. If they are established data centres you will almost certainly find that there is existing ducting between the two buildings, and if you are lucky they may already be spare capacity for sale. If not it's not a massive task to blow some more fibre through the existing ducting.
Another option would be to try and find an ISP that has presence in both DC's. It would seem unlikely that in such a scenario traffic between the sites would go via a 3rd POP - and that would satisfy your latency concerns.
You'll run into problems if that ducting does not already exist though because your options will be to either go via a third POP, which would probably not meet your latency requirements, or to have someone dig up the road, which would drive the install cost through the roof.
There really isn't given the conditions you've applied. The cheapest way to get between two data centers is wireless, however this will add a significant amount of latency (3+ms) to the connection. You can get faster wireless solutions, but in my experience that means they will need to be band licensed/microwave and the hardware for both sides will probably cost ~$20,000 and you'll have to either rent spectrum from someone or get it yourself from the FCC. The fastest way (latency wise) to connect them is to trench fiber between the two, costs will vary for this but if there isn't conduit already existing it can be in the tens of thousands to do this, just for 100m.
So to exactly answer your question, no there really isn't a cost effective low latency solution. That being said, if ~3-6ms isn't a huge deal you could probably rent roof space at both DCs (usually this can be done for a small fee, in the 100s/mo) and get long range carrier grade wifi gear to connect them (will probably be 2-3k$). If that latency is really too much you'll probably be stuck trying to find someone who has a Metro Ring in the area and seeing if you can hop on/off between the two facilities without having to route through a CO somewhere. That would be much lower latency, but probably pretty costly depending on how much bandwidth you need.
You had some other questions in there about what causes latency, that is probably best saved for a separate question as it applies to general networking. However to briefly answer it, most of the time it is the speed of light. Once you enter a facility the switching and routing is usually done at wire speed and thus doesn't add a significant delay. There are some edge cases like DSL and wireless that have serialization delay where the signaling done on the wire adds some significant overhead, but this is uncommon in data center situations where its usually SONET/ATM or ethernet.
To give an example in a well connected exchange like an Equinix data center you can get to any ISP in there and outside of the building in <1ms.
The cost for a data center 100 metres down the road is substantially less
Yes, and guess why - because it has higher latency. Decide and eat it. You wont play HFT games with a oom 100 meters down the road.
THere are specialised hosting oeprations that give you singledigit ms acess to exchagnes. I was quoted 150 USD per month for a 1u server within 3ms of the CME group... and 1500 for something withless than 1ms (significantly).
Now, no whining will make the 100 meter comparable to the expensive offer.... light speed and stuff slow signals down. HFT is a money game, wuere eople buy from specialized companies deep frozen cpus highly overclocked and run special linux kernels that are optimized for network latency.
