2

Just wondering there is any kind of stress test I can put the ASA through. I'm asking this not to help with a problem but just for reference.

I know I could use iperf to send a bunch of traffic through it but was thinking there might be something more I could do.

evolvd
  • 1,384
  • 6
  • 33
  • 58

2 Answers2

5

For a firewall device It is quite common to stress test the throughput capacity and simultaneous connections (and in some scenarios even the maximum size of the NAT translation table).

Then there is the amount of new connections per unit of time the firewall device can establish (rate of new connections).

This parameter is of huge significance in a firewall and in my experience, not many people talks about it and as a consequence, not many people do test it.

In fact, a couple of times I have seen firewalls fell on their knees under a DDoS that is well under the maximum advertised throughput of the firewall but which has a very high connection rate (last I remember above ~100Kconns/sec).

When this happens the explaining is not always easy to do:

"What do you mean with a 100Mbps DDoS our firewall fell???, isn't it supposed to handle Zillions of Gbps of throughput??? I want to talk with the sales engineer right know dammit!"

jliendo
  • 1,578
  • 11
  • 13
  • 1
    Personally I don't see the value in stress testing a firewall for it's ability to handle situations that don't reflect real usage. What's the value in testing it's ability to handle a DDOS attack? Do you want it to continue handling normal traffic while under a DDOS attack? I don't crash my car into a wall to stress test it's airbag. Why the need to stress test it at all? Do you stress test the fire retardent material in your house by lighting a fire in the living room? I'm trying to understand the logic here. Stress testing it's ability to handle your projected traffic load is valid though. – joeqwerty Apr 05 '11 at 17:43
  • In addition, how can you stress test it for a situation you have no ability to quantify. If the firewall comes under attack how can you stress test it before the attack for it's ability to withstand the attack without knowing the volume and scope of the attack? – joeqwerty Apr 05 '11 at 17:44
  • 1
    @joeqerty: "how can you stress test it before the attack .. without knowig the volumne and scope.." Well, that's kind of what stress tests are for. To determine the volume and scope of what can be handled by a product BEFORE failure. In other words the stress test should STRESS the product to find the point of failure. – NotMe Apr 05 '11 at 18:49
  • @Chris: Good points. My point being that you should purchase a device based on it's documented (from the device specifications) ability to handle the load that you expect it to handle and for your projected usage of said device. I don't crash my car into a wall just to prove that the manufacturer's safety ratings hold true. I also don't drive my car at 120MPH to see how far I can push my 80MPH rated tires. Maybe "load testing" or "usage testing" are better terms than "stress testing". – joeqwerty Apr 06 '11 at 00:08
  • 1
    @joeqwerty: The thing is, someone actually did crash the car in question in a wall to make those safety ratings. Usually they do 7 of them just to be sure. Regarding computer products it's buyer beware and you can't trust anything a manufacturer says. NIC manufacturers advertise 1GB/s, but you'll never actually see that rate. HD Manufactures tout the new 6 GB/s interface, but there isn't a single drive on the planet that can transfer that fast. I could go on, point is it's called stress testing for a reason. – NotMe Apr 06 '11 at 02:57
  • @Chris: Again, good points. Of course the manufacturer puts the product through stress tests, so that the consumer doesn't have to. When you purchase a product it's already been determined (by the manufacturer) what said device can and can't do and what it's limitations and capabilities are. You shouldn't have to stress test a device, that's what device specifications are for, so you as the consumer know before making a purchase whether or not the device suits your needs and usage. If you have to stress test it you probably haven't done enough to determine what your needs and usage are. – joeqwerty Apr 06 '11 at 11:44
  • Do you stress test hard drives, NIC's, memory sticks, and CPU's when you purchase them? Maybe I'm hung up on semantics here. Load testing is valid but I see no point in stress testing. Stress testing implies that you're going to push a device to and beyond it's rated limits, which is pointless in my opinion. The manufacturer has already done that. Load testing to determine how well a device operates at the projected usage is valid. – joeqwerty Apr 06 '11 at 11:50
  • @joeqwerty: Actually that testing is done by the government (NTSB). The vendors might crash their own cars, but the NTSB performs their own crash tests on behalf of the people. In other words, the words of the vendor arent' trusted and instead are independently verified. – NotMe Apr 06 '11 at 15:39
  • @joeqwerty: With regards to stress testing components: we absolutely do that along with load and performance testing. There is a HUGE difference in how a given product fails between manufacturers. Further the manufacturers I see that advertsise the results for this are for hard drives; it's used to come up with a MTBF rating. That said, Intel and others have started stress testing full servers in order to determine the real cooling requirements for data centers. It's interesting to see how the actual numbers are different from what's been advertised. – NotMe Apr 06 '11 at 15:45
  • @joeqwerty: One last thought. In this particular case I would want to know what happens when the device was completely overloaded. Does it simply stop accepting new connections or is it worse. For example: does it drop current connections, shut down, lose the magic smoke, or worse, simply pass through all connections. 15 years ago you could overload the Cisco PIX and it would stop protecting your network and pass all connections through. We found this by stress testing the device. Cisco sure wasn't going to give us this info. – NotMe Apr 06 '11 at 15:54
  • @Chris: Thanks for the well thought out comments. I'll respectfully agree to disagree with you, but I appreciate your point of view on the issue and the time you've taken to state your position. – joeqwerty Apr 07 '11 at 03:23
4

Kudos for even thinking about stress testing. In my experience you really have to break down the components of the firewall and design your tests for specific circumstances depending on your policies and configuration. @jliendo pointed out some good items for consideration. Connections per second instead of merely throughput is important to think about and you could try a real-world test of that by downloading a heavily seeded torrent. Your device could also have the IPS card installed so if it did, you'd want to thoroughly test stress any policies that you setup for packet inspection with the sensor. Did you setup Antivirus scanning? You would want to test throughput of that policy to see if it can keep up. In general, the goal of a stress test would be to test your policies more so than the raw power of the hardware; since the policies can significantly impact the performance levels of the hardware.

With the above in mind, you would find a collection of tools such as iperf to help you analyze specific cases. Iperf can help you test your storm control policies. Seagull is a useful traffic generator that can test most of your other policies. (Remember to test whether or not it works as intended, AND whether it blocks traffic as intended) Depending on your environment you may also want to test latency and poor network connections (perhaps the 5505 is a branch-office firewall?) to help you adjust your timeouts. The trial version of Netlimiter is very helpful in this regard.

In summary I would re-approach your question from the view of your configuration. If you have configured the box to allow or deny traffic in a certain situation, then ask yourself how to test whether or not that works as intended. Once you tested at this level, then try to test multiple policies (such as HTTP through the AV Scan to a secondary subnet on a VLAN). Then bomb the device to death with a traffic generator, find weak points in the flow, and setup quality of service to protect yourself in the case of extreme situations.

Christopher
  • 111
  • 3