Potential traffic issues on one big(ger) subnet

Question

Posts like this:

Most traffic in a corporate LAN is client to server nowadays, and a router not very well setup rather becomes a bottleneck AND a SPOF. and 200+ clients on a subnet wasn't a real issue 10 years ago and it won't be now, you can still read all the broadcasts with a (non-promisc) tcpdump without it becoming a blur - tiny compared to the bandwidth. And ARP work for the clients should also be no issue nowadays. – rackandboneman May 21 '12 at 19:34

...and this:

If all of you have is 50 clients, then it would not make a difference if the subnet was /8 or /24. Its the same number of clients, same amount of traffic. In any event, subnetting your network is not really based on the number of computers, but the need to segregate the systems based on security requirements, traffic isolation, etc...

...together seem to contradict advice I had been given from a network professional, who told me that my current 10.0.0.0/8 subnet (with about 20 clients and 2 servers all hooked to the same switch) was vulnerable to overloading should a client be compromised by malware, because broadcast traffic would be orders of magnitude higher than, say, on a 192.168.0.0/24 subnet. Is this perhaps what the poster of the above quote was referring to? Or is the advice out of date?

Answer 1

It has been a while since I worked in the network administration side of things but here is what I think about this:

200+ clients on a subnet wasn't a real issue 10 years ago and it won't be now, you can still read all the broadcasts

This is more or less true. Back before switching technology subnetting to account for the total amount of broadcast traffic was a serious concern. Now days though switches are much more efficient both in terms of general architecture compared to hubs (i.e., they do not forward EVERY packet to EVERY port) and they are better with the broadcast traffic they do have.

I have heard WAG numbers that 500 clients per subnet is about where you should start to consider subnetting based solely on traffic and broadcast domain concerns but I would not be surprised if enterprise-grade switching hardware could handle much more. Obviously, test and test again as everyone's workload is different.

you can still read all the broadcasts with a (non-promisc) tcpdump without it becoming a blur

If your IDS/IPS requires you to read broadcast traffic manually then you probably should look at a different IDS/IPS product. I do not really see this as a valid concern in deciding what your subnet size should be.

If all you have is 50 clients, then it would not make a difference if the subnet was /8 or /24. Its the same number of clients, same amount of traffic.

Seems logical to me. The network space aside, you only have so many clients and they can only produce so much traffic.

my current 10.0.0.0/8 subnet (with about 20 clients and 2 servers all hooked to the same switch) was vulnerable to overloading should a client be compromised by malware, because broadcast traffic would be orders of magnitude higher than, say, on a 192.168.0.0/24 subnet

Wow. I would right-size your subnet, like right now!

I am going to copy/paste this from my other answer but it is very relevant here: You are not managing hundreds of hosts. The complexity of your solution should reflect the complexity of environment. Resist the temptation to be overly clever. You will thank yourself later.

Second, I am not sure how it would be more vulnerable to overloading with regards to broadcast traffic since there's only 20 clients to broadcast. When you think about broadcast attack or broadcast fan-outs the limiting factor is generally not the broadcast domain but the the nodes generating the traffic so if your 20 nodes are attempting to broadcast to 252 IP addresses or 16,777,212 IP addresses ( 16,777,192 of which are unoccupied) the same number of broadcasts are going out. Now if the malware does some kind of amplification attack where it starts creating IP addresses, yeah, you have given your attacker a lot more room to play. Maybe that's what your security guy was getting at. Information Security is complicated and I only have a cursory knowledge of the field so if you want to explore this question in more detail perhaps Security.SE would be more appropriate.

Answer 2

There are a lot of "it depends" that go with this question.

Security

Broadcast storms, floods and the ilk are certainly a concern if you are using consumer network gear. A majority of enterprise network equipment have ways to deal with this however. Depending on your vendor, you may have options for "Storm Control". As with anything, these options require testing and can also have their own operational behavior (both good and bad).

Another aspect of security is around the ability to separate systems by their role and keep traffic isolated. This concept is also evolving with time, especially around SDN, virtualization, etc. VLAN segmentation by itself may or may not be sufficient security isolation, depending on the needs of your organization. There are far too many conditions specific to your org to answer this correctly.

Network Size

A /8 for 20 hosts is a bit large, but technically there is nothing stopping you from doing this. My concern would be network growth. At some point, you may want to connect this network to other networks, data-centers, offices, etc. By using the entire /8 at your location, you would have to NAT all your traffic to reach other networks in 10/8. Typically, folks will size their networks so that they can allocate a "standard size" subnet to each location and use a WAN or VPN mesh to interconnect them, having firewall rules at each location that allow specific services to reach specific destinations based on roles, etc.

If you actually start filling up a large LAN then at some point you will need to increase the limits in your servers and workstations to allow for larger ARP tables and minimize garbage collection of ARP entries to avoid excessive ARP updates. The same is true for all the access and distribution switches in your network. Each server/workstation OS and each network vendor will exhibit different behavior as you start populating a very large broadcast domain.

My own experience with large VLANs

The largest fully populated network I have been involved with was a /22. The only issues we ran into were services that depended on fast responces from multicast traffic. Even increasing the ARP limits in the servers was not enough to address this. We had to resize our networks into smaller VLANs and change the configuration of our applications to depend less on multicast.