What does 'tcp[12] & 80 != 0' do?

Question

I realize that tcpdump[12] refers to the header length field. But what does '& 80 != 0' refer to?

Answer 1

If you refer to the various TCP-related RFC's, beginning with RFC 793 (the original) as well as at least RFC 3540, the one that introduced ECN, you will see from Section 5 of RFC3540 that the 13th byte (offsets are 0-based) in a TCP header is made up of the 4 bits that comprise the TCP Header Length, the 3 bits of the Reserved field, and the NS (or nonce sum) bit. Specifically, tcp[12] is looking at these bits:

  0   1   2   3   4   5   6   7
+---+---+---+---+---+---+---+---+
|               |           | N |
| Header Length | Reserved  | S |
|               |           |   |
+---+---+---+---+---+---+---+---+

The & operator is the bitwise AND operator, although surprisingly this doesn't appear to be documented in the pcap-filter man page. It takes the byte specified by tcp[12] (described above) and performs a bitwise AND operation with the value of 80 (or 0x50 or 01010000 binary), which is then compared against being non-zero using the != 0 operation.

Since 01010000 only has 2 bits set, this has the effect of checking if the TCP Header Length field contains a value such that either bit 1 or bit 3 in the above diagram is set. Since the Header Length field represents "The number of 32 bit words in the TCP Header." essentially this means that the filter will return TRUE (and thus match the packet) if the TCP Header Length field contains any of the following twelve 4-bit sequences, and effectively the following TCP Header Length value (in bytes):

Header Length  | Effective Length (in bytes)
---------------+---------------------------
 0  0  0  1    | 4    (1 * 4)
 0  0  1  1    | 12   (3 * 4)
 0  1  0  0    | 16   (4 * 4)
 0  1  0  1    | 20   (5 * 4)
 0  1  1  0    | 24   (6 * 4)
 0  1  1  1    | 28   (7 * 4)
 1  0  0  1    | 36   (9 * 4)
 1  0  1  1    | 44   (11 * 4)
 1  1  0  0    | 48   (12 * 4)
 1  1  0  1    | 52   (13 * 4)
 1  1  1  0    | 56   (14 * 4)
 1  1  1  1    | 60   (15 * 4)
---------------+---------------------------

Now the question is, "Why would someone want to check the TCP Header Length field against these twelve specific values?" That is a good question for which I have no answer. I would be willing to bet that the filter is not the intended one though and that instead of 'tcp[12] & 80 != 0', the intent was to use 'tcp[12] & 0x80 != 0', which would effectively test for any TCP Header Length field that is greater than or equal to 32 bytes in length (i.e., 32, 36, 40, 44, 48, 52, 56 or 60).

In case you're not aware, 0x indicates a hexadecimal number so 0x80 is decimal 128 and in binary, this is 10000000 so this only checks if the most significant bit of the Header Length is set to 1, which it is when the TCP Header Length field is one of the eight values listed above starting with 32.

P.S. Another reference that I like to use for various protocols is provided by a company called Inacon, some of which, like TCP, are free to reference.