1

There are advantages when you can deploy one server A so that it serves as the IP gateway for machines B, C, and D. In a real machine room, for example, A can balance incoming customer connections by forward packets to B, C, and D without even pausing to rewrite them, so long as B, C, and D are all programmed to think of the service's public IP address as their own — and all reply packets can flow directly back to customers without returning through the load balancer.

Less ambitiously, something like HAProxy can be used with the Linux Tproxy kernel extension on the gateway machine A, so that incoming load-balanced connections can have their packets forwarded to B, C, and D without changing the source IP address, but only the destination IP address. While this requires return packets to be rewritten, and B, C, and D have to be configured to use A as their IP gateway, it solves the problem of how to load-balance without losing the ability to detect and log customer IP addresses on servers B, C, and D.

My question, then:

Which cloud hosting providers allow machines to be placed on a virtual LAN such that real link-level freedom is provided to send packets between machines simply based on MAC address, and without regard for IP addresses?

If machine B is going to use machine A as an IP gateway, then obviously machine A will be receiving lots of packets from machine B that do not have A's IP address, but the address of other hosts out on the Internet with which B wishes to communicate.

I spent an hour last night experimenting with the Virtual Private Cloud (VPC) service offered by Amazon, but found that having machine A serve as a gateway was not possible — although I could tell B that A was its IP gateway, and turn on ip_forward on A, the packets that B tried to send off-LAN never made it to A, apparently because Amazon requires both the MAC address and the IP address to be correct before passing a packet from B to A.

Which (if any) cloud hosting providers would let me set up a LAN so that machines can pass packets purely based on MAC addressing?

Brandon Rhodes
  • 547
  • 4
  • 6

1 Answers1

1

You can work around this by using openvpn and ethernet bridging to create a virtual layer two segment shared by all of the servers.

mfarver
  • 2,576
  • 14
  • 16
  • An interesting idea, @mfarver! And if I were not worried about my in-cloud network packets being captured by other customers, could I also construct a solution like this using Linux's native IPIP tunneling, imposing less CPU load? – Brandon Rhodes May 30 '11 at 19:02
  • 1
    IP/IP is Layer 3, I think. You said you wanted Layer 2. OpenVPN is unique because it can use the TAP interface to give you a L2 connection between two servers which you can bridge to other Layer 2 tunnels. You can tell OpenVPN to not use encryption if you want to save the overhead: http://wiki.mikrotik.com/wiki/OpenVPN#Disable_encryption – mfarver May 30 '11 at 22:43
  • You are right, of course — IPIP is layer 3. I only mentioned it as a possible alternative to accomplish what I was aiming for: to get Internet-bound packets to and from a “gateway” without Amazon being offended that their IP destination was not that of the gateway itself. Thanks for the OpenVPN link — I had *never* thought of using it for features besides the encryption! (Since I am the first one to ask this question on Server Fault, I cannot help thinking that maybe I am going at this whole problem wrong? How do other people do load balancing without losing customer IP addresses? Not Amazon?) – Brandon Rhodes May 31 '11 at 03:59
  • One other question, @mfarver: does OpenVPN encapsulation tend to cause any MTU-related problems, as incoming packets have to be fragmented to fit the last hop from one LAN machine to its peers? Or are efficiency and service not harmed by the limits of that last hop? – Brandon Rhodes Jun 01 '11 at 11:47