6

Border topology for largish enterprise http://www.freeimagehosting.net/uploads/d24ede3b2f.png

Let's say you've got an enterprise with a small internet presence in the form of a DNS server, a web server, and a VPN server, all on the DMZ (in this case, an invisible pair of switches connected directly to the firewalls). There are also 20,000 nodes inside the enterprise network, spread across lots of buildings, with lots of routers and vlans and such. The enterprise is full of very important, very busy people who like to be able to get to facebook without delay... Everything gets natted; the DMZ has static symmetric NATs and everyone else gets to share outgoing IP addresses from a pool set aside for this purpose.

The border is a pair of routers connected to each other and the firewalls via a pair of gig ethernet switches, and to the ISPs via metro ethernet and are connected to the ISPs via metro ethernet. The border routers exchange routes with the ISPs via eBGP and with each other via iBGP. The interior network has diverse routes to any point in the network and uses a dynamic routing protocol to manage failover and route distribution.

The firewalls are connected to the enterprise core via a pair of routers in the datacenter.

My question is this:

Static routes or dynamic routing protocols?

  • static routes:

    Each set of devices has static routes and uses some mechanism like VRRP or HSRP to manage the L2<->L3 failover. The firewalls would point to the virtual address for the border routers for the enterprise default route, the interior routers would point to the virtual address of the firewalls for their default route, and the firewalls would point to the virtual address of the interior routers for 10.0.0.0/8.

  • dynamic routing:

    Use iBGP between the firewalls and the border routers and OSPF / EIGRP between the firewall and the interior routers.

I've seen that people use the static route model far more often than the dynamic routing model. My question is -- why?

What are the best practices for this sort of topology? Or is this just a religious sort of issue?

chris
  • 11,944
  • 6
  • 42
  • 51
  • The image you are linking to seem to have been removed. Could you maybe paste the image directly into your post instead? – Deantwo Aug 16 '18 at 06:29

3 Answers3

1

Put external routers into HSRP/VRRP pair. Get an AS. Start doing BGP with your ISPs.

Then, everywhere inside your environment, the paths will be static. You just need to have failover, and that's what's achieved with VRRP/HSRP and firewalls in A/A or A/S configuration.

Alternatives are messier and/or less robust:

  • Dynamic throughout is an unnecessary hassle.
  • Static outbound on the firewalls would make it difficult/inflexible/impossible to use both uplinks simultaneously.
  • Inbound redundancy will be a world of pain without kludges like PePLink/Ecessa/EdgeXOS.
Max Alginin
  • 3,284
  • 15
  • 11
  • Let's assume that the enterprise has a public AS... Two questions: Can you use HSRP if each border device is only connected to one ISP? With either, if an upstream route or interface fails, the virtual IP needs to flap over to the other external router, and to achieve that tracking you need all sorts of kludges like "flap if you can't ping this external IP or if this interface goes down or this BGP session goes down." and each of those has edge conditions where they don't "do the right thing" – chris Dec 11 '09 at 11:27
  • Oh -- and with static outbound routes -- if you're not using iBGP between the firewall(s) and the border routers -- don't you *have* to use a static route on the firewall to the virtual IP of the border router pair? And then, if you want to do more routing, the traffic would be sent back through that internal link to the *other* border router, which may cause congestion at that link. – chris Dec 11 '09 at 11:29
  • You have to connect both border routers to both ISPs - just insert a pair of switches between your uplinks and routers and configure another VIP on the uplink side. And yes, there will be a single static route on the firewall to the border's inside VIP. – Max Alginin Dec 11 '09 at 15:27
1

Note: all of the below assume you have a single site, with two ISP circuits. Please let me know if that's not the case.

For routing between your enterprise core and your DMZ, it sounds like static routing is perfectly appropriate; default route pointing to the firewalls, a route for your internal subnets (10.0.0.0/8, from the sounds of it), back to your core routers. (Although there is a school of thought that says the internet should not be routeable from your core network; more on that in an addendum at the end.) That said, if you summarise neatly, there would not be much overhead in running OSPF, EIGRP or even EBGP between your core and your DMZ. It's often down to the personal preference of the engineer.

Regarding routing between your DMZ and your providers, there are two aspects:

  1. How your ISPs route to you
  2. How you route to the internet via your ISPs

Regarding (1): do you have provider independent address space, allowing your VPN server and web server to be presented by both ISPs? Assuming this is the case, I would suggest that running EBGP to your providers is not a bad idea - it allows you to influence which ISP is preferred for your incoming traffic, through the use of various BGP attributes. If your servers are on provider specific address space, then there is no benefit to dynamically advertising routes to your ISPs; they may as well statically route to you.

Regarding (2): a default static route to an HSRP/VRRP address on your edge routers will work; you can use object tracking (on Cisco platforms) to determine the 'health' of the locally attached ISP circuit on each router, perhaps by pinging a well known IP. If the tracked object goes down, the router can be configured to decrease its HSRP/VRRP priority, failing traffic over to the other link.

That said, using a default is very course grained, and does not allow you to make use of any direct peerings your ISP may have with big content providers (e.g. akamai, BBC, etc). If this might be of use, EBGP peering will allow you to get more visibility into your ISPs connectivity. Many offer standard communities that allow you to accept 'customer' routes, meaning you have specific routes for any directly connected customers of your ISP, but do not need to maintain a copy of the full internet BGP table.

An aside: looking at your description, it seems that one piece of infrastructure that might be missing is an internet proxy. There is a lot to be said for not allowing your internal hosts to route directly to the internet; if the majority of outbound traffic is only web browsing, by installing one (or two, for failover) proxy in your DMZ, you could remove the need for your internal network to have a default route to the DMZ. Simply route to the DMZ subnets (where the proxy resides), and have internet reachability limited to your DMZ. A couple of benefits:

  1. Centralised policy control; it allows you to restrict access to sites that your management/security people view as either inappropriate or high risk.
  2. It limits the ability of any malware that makes it on to user workstations to 'phone home'. Anything that attempts to route directly to internet addresses will not be able to connect; anything that attempts to use the proxy will have to use only permitted ports (80/443), and if you subscribe to a good blacklist provider, your proxy can reduce your exposure even further.

Great question. :)

Murali Suriar
  • 10,296
  • 8
  • 41
  • 62
  • Two providers, the enterprise has its own AS and associated address space. That said -- the issue of diverse routes -> static routes -> diverse routes still comes into play with interior -> exterior connectivity even if you can do some L2 failover at the ISP edge. – chris Dec 11 '09 at 14:13
  • I absolutely agree; there are a number of failures that static routes simply will not catch, which is why my preference is to run a dynamic routing protocol of some sort wherever feasible. That said, many people like the simplicity of static routes to face off to DMZs/providers; I suppose it depends on how comfortable you are with large scale routed environments. I'll update my answer to cover some of the edge cases this weekend. – Murali Suriar Dec 11 '09 at 21:55
0

The static approach allows a more fine-grained control on how the traffic is routed to your two ISPs. You can have asymmetric routing depending on the speed of your ISPs or depending on what services you want to access.

The replication/fail-over is fairly simple to set up, as most firewall/appliance offer very simple procedure (as least on Cisco).

Laurent Etiemble
  • 246
  • 2
  • 7
  • About the firewall failover -- that part is typically pretty simple. The hard part is mixing static routes on the firewall and dynamic routes into and out of the firewall (ie if there are 2 potential default gateways into the interior network or into the internet). Things like VRRP have nasty edge cases where you can lose connectivity into or out of the dynamically routed part of the network. – chris Dec 11 '09 at 13:30