5

On a busy† Debian Lenny server of mine I am considering disabling TCP Window Scaling. Why?

  • I'd like to enable syn cookies, which disables TCP Window Scaling. This server is locally firewalled and the protection against syn flood attacks could be a good thing, right?
  • The kernel logs have many "TCP: Treason uncloaked!" messages. It does not appear to be an attack, having occurred regularly for ages, but it still concerns me. From what I understand, this message is a result of a disagrement between client and server about the size of the TCP window and is generally not a big deal.

So I ask myself "Does the box really need TCP Window Scaling?" Before I even try to experiment and benchmark, it seems prudent to query the boffins of ServerFault.

Some relevant details:

  • many (10-30%) of requests are for 5-50MB files
  • the large files are sent at a regulated bit rate (~2Mbps)
  • clients are on the Internet and 90% within 250km

How valuable is TCP Window Scaling?

  • Is the CPU dramatically impacted? If so, how much?
  • Is networking performance degraded? Latency does not bother me, but throughput below a minimum threshold would.
  • Anything else I might be missing?

† = 3Gbps worth of LACP'd NICs + hundreds of millions of HTTP requests + tens of terabytes of traffic per month

Community
  • 1
Stu Thompson
  • 3,349
  • 7
  • 31
  • 47

3 Answers3

4

"Does the box really need TCP Window Scaling?"

Well, you didn't really say what the box does, so it's kind of hard to say really. But in general, TCP Window Scaling is critical for good end user performance on modern WAN/Internet connections. It's less needed on LAN.

Is networking performance degraded? Latency does not bother me, but throughput would.

It depends on your users' network connections. But if some of your users are

  • physically far away from you (i.e. have high latency), and
  • have fast network connections (i.e. good DSL, fiber, etc)

... then without Window Scaling, these users will only be able to use a small fraction of their connection speed when downloading from you. Here is a good tutorial on bandwith-delay product and window sizes -- complete with nice animated cartoons.

"Enabling High Performance Data Transfers" is a classical document about TCP window scaling. It correctly mentions that if you're using a recent Linux kernel in the 2.6 series, then you normally don't need to tweak the TCP settings, because Linux now has agressive auto-tuning of these. It fails to mention that Windows 2008+ also auto-tunes TCP settings well, using Compound TCP.

Update:

the large files are sent at a regulated bit rate (~2Mbps) clients are on the Internet and 90% within 250km

With this updated info, it gets trickier. Since you are capping the maximum speed, maybe TCP windows are not a limitation in your case. Have a look at what kind of connections your end users are using, and do the math. You don't need to run benchmarks, you can calculate the required window size, and compare it to your servers actual default transmit window size.

  • Thanks. I'm hesitant to experiment on a production box. Need to think on it first. It sounds like if I were to kill the window scaling then users farther away might suffer...so maybe my benchmarks for an experiment needs to focus on them. Hmmm... – Stu Thompson Apr 07 '11 at 10:23
  • @Stu Thompson: Mmn, why experiment? Just grock bandwith-delay product, and default TCP Transmit Window Size (64k on many OS's), as per @Javier . Then do the math. Something else though, for me it seems wrong to tweak default OS TCP settings just to obtain Syn Cookies support - the benefit is likely to be small, as SYN DDoS attacks are just one out of many possible DDoS attack vectors. If you really p*ss off a large modern DDoS botnet operator, then Syn Cookies alone won't save you. Switch on Syn Cookies only if/when you face a SYN DDoS attack? –  Apr 07 '11 at 12:38
  • Ah, good idea about just being reactive to an attack. I'll have to set up something to know when/if it happens. But I've also got the *"Treason uncloaked"* stuff. Note I've had to tweak my TCP settings from the the default already, but that is unrelated. (`nf_conntrack_max` related stuff.) – Stu Thompson Apr 08 '11 at 05:00
3

In many cases, TCP autothrottling means that a given transfer is limited to:

BW = windowsize / RTT

since old TCP standards limited the maximum window size to 64K, you could easily get a "Long Fat Network" (LFNs, or 'elephan') if that figure was lower than the physical bandwith. Window scaling to the rescue!

To check if you need it, do some packet capturing and analyze the RTT (don't rely too much on ping times, WireShark has a tool to do that on a sample of real traffic). The average throughput a user would see if you turn it off would be 64KB/RTT (in bytes/sec).

Javier
  • 9,268
  • 2
  • 24
  • 24
2

Is networking performance degraded?

Yes - but only for transferring files larger than RTTxBW. But when you start hauling around big files on a long fat network the performance degrades very quickly.

So whether it affects you will depend on what you're serving (you didn't say).

symcbean
  • 21,009
  • 1
  • 31
  • 52
  • Thanks. I've head about *"long fat networks"* but do know know if that applies to me or not. I've updated the question with more details. – Stu Thompson Apr 07 '11 at 09:36