Can I use tcpdump to get HTTP requests, response header and response body?

Question

I am using tcpdump to get HTTP data by executing the below command:

sudo tcpdump -A -s 1492 dst port 80

The result of above command:

Headers, I think request and response headers.
Unreadable data.
The url GET /modules/mod_news_pro_gk1/cache/stories.ilbalad.ajayeb.strange-tractor.jpg.

I need a more clear result, for example, readable request > response header > response body etc. How can I filter my results?

HTTP responses may be compressed or chunked and getting their raw text may not really do what you need ("unreadable data"). You probably need to explain more completely what you want to do and what your environment permits (e.g. can't you just use a HTTP proxy like Charles or Fiddler)? — EricLaw, Jan 23 '11 at 22:50
how did you run `tcpdump` on Android OS? Do you somehow embed the command in your app or you run it on a PC that is connected to the Android phone? — faizal, Jul 25 '14 at 07:13
@faizal - you should install the tcpdump binary file into the device, then you can run it from the device itself, please use this link for How to install tcpdump into Android device : http://gadgetcat.wordpress.com/2011/09/11/tcpdump-on-android/ — kimo, Jul 25 '14 at 21:33

score 236 · Accepted Answer · edited Mar 09 '20 at 08:42

236

There are tcpdump filters for HTTP GET & HTTP POST (or for both plus message body):

Run man tcpdump | less -Ip examples to see some examples

Here’s a tcpdump filter for HTTP GET (GET = 0x47, 0x45, 0x54, 0x20):

sudo tcpdump -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'

Here’s a tcpdump filter for HTTP POST (POST = 0x50, 0x4f, 0x53, 0x54):

sudo tcpdump -s 0 -A 'tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)'

Monitor HTTP traffic including request and response headers and message body (source):

tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
tcpdump -X -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

For more information on the bit-twiddling in the TCP header see: String-Matching Capture Filter Generator (link to Sake Blok's explanation).

edited Mar 09 '20 at 08:42

Per Lundberg

3,837
1
36
46

answered May 17 '13 at 13:31

paulz

2,376
2
13
2

18

I didn't know about `less -Ip` - what a timesaver – n611x007 Nov 07 '13 at 12:36
11

I'd like to see an explanation of those tcpdump invokations – tmc Jan 31 '14 at 03:17
1

Is it possible to have a filter for HTTP GET and POST together? – Jim Ho Feb 04 '16 at 04:01
4

This is sorcery. Someone please explain how it works. – Jul 14 '17 at 07:40
1

https://explainshell.com/explain?cmd=sudo+tcpdump+-s+0+-A+%27tcp%5B%28%28tcp%5B12%3A1%5D+%26+0xf0%29+%3E%3E+2%29%3A4%5D+%3D+0x47455420%27 – jmunsch Dec 14 '18 at 20:32
What is the filter for PUT and DELETE? I tried with PUT 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x50555420' and 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x44454c455445' for PUT and DELETE respectively, for PUT it is working but somehow it did not work for DELETE. Am I missing something? – Praveen Patel Oct 17 '19 at 07:48
`0x504f5354` is `POST` in ASCII encoding, written in hex. Likewise `0x47455420`, is `GET ` (with trailing space). The invocations above utilize the fact that in the HTTP protocol, the request starts with the HTTP method, so by checking the first 32-bit word against these hex values, we can easily check if the request matches a given HTTP method. – Per Lundberg Mar 09 '20 at 08:40
Added some of this into the post to make it be slightly easier to understand. – Per Lundberg Mar 09 '20 at 08:42

score 33 · Answer 2 · edited Sep 18 '15 at 14:50

33

I would recommend using Wireshark, which has a "Follow TCP Stream" option that makes it very easy to see the full requests and responses for a particular TCP connection. If you would prefer to use the command line, you can try tcpflow, a tool dedicated to capturing and reconstructing the contents of TCP streams.

Other options would be using an HTTP debugging proxy, like Charles or Fiddler as EricLaw suggests. These have the advantage of having specific support for HTTP to make it easier to deal with various sorts of encodings, and other features like saving requests to replay them or editing requests.

You could also use a tool like Firebug (Firefox), Web Inspector (Safari, Chrome, and other WebKit-based browsers), or Opera Dragonfly, all of which provide some ability to view the request and response headers and bodies (though most of them don't allow you to see the exact byte stream, but instead how the browsers parsed the requests).

And finally, you can always construct requests by hand, using something like telnet, netcat, or socat to connect to port 80 and type the request in manually, or a tool like htty to help easily construct a request and inspect the response.

edited Sep 18 '15 at 14:50

dotancohen

30,064
36
138
197

answered Jan 23 '11 at 23:33

Brian Campbell

322,767
57
360
340

2

Thanks a lot for your answer Brain. I am using tcpdump in android OS :) therefore i am limited by using tcpdump,for sure I can get the headers and also the link request, i need also to get the body, any help, any Perl convert script or something like that? – kimo Jan 24 '11 at 11:32
11

@kimo In that case, I'd recommend using `tcpdump -w outfile ...` to dump the raw packets into a file, and then copy it to your computer, and use Wireshark to analyze it. Wireshark can open up a packet dump, decode the TCP stream, and decompress the HTTP content (I'm assuming that the reason it's unreadable is because it's compressed). – Brian Campbell Jan 24 '11 at 18:37
Thanks Brian, i will try to implement this solution, anyway it's better to move the file to PC, because i should filter the data to get the wanted requests, and it is not easy to do it on android system. – kimo Jan 24 '11 at 23:23
Brian I think what we are trying to say here is. How can we foo this with Busy Box. I am having the same constraints on Android devices also. – Tegra Detra Nov 23 '13 at 03:28
1

@JamesAndino Yes, on an Android device, I would recommend using `tcpdump -w outfile`, then copying that to a machine that has Wireshark and using Wireshark to view the dump. – Brian Campbell Nov 23 '13 at 06:14
There is a Firefox addon called [Live HTTP Headers](https://addons.mozilla.org/En-us/firefox/addon/live-http-headers/) that will give you the entire HTTP request and response. It's also [available for Chrome](https://chrome.google.com/webstore/detail/live-http-headers/iaiioopjkcekapmldfgbebdclcnpgnlo?hl=en) – SKWebDev Mar 25 '15 at 19:24

score 7 · Answer 3 · edited Jun 15 '12 at 03:20

Here is another choice: Chaosreader

So I need to debug an application which posts xml to a 3rd party application. I found a brilliant little perl script which does all the hard work – you just chuck it a tcpdump output file, and it does all the manipulation and outputs everything you need...

The script is called chaosreader0.94. See http://www.darknet.org.uk/2007/11/chaosreader-trace-tcpudp-sessions-from-tcpdump/

It worked like a treat, I did the following:

tcpdump host www.blah.com -s 9000 -w outputfile; perl chaosreader0.94 outputfile

score 4 · Answer 4 · answered Oct 27 '22 at 22:38

4

This is what I tend to use:

tcpdump -s0 -Aql host example.com and port 80 -i any | grep -A5 -P 'HTTP'

Where -A5 is the number of lines you want to match after HTTP.

answered Oct 27 '22 at 22:38

KJ7LNW

1,437
5
11

1

Since HTTP headers tend to end with a blank line, something like `sed '/HTTP/,/^$/!d'` might work instead of `grep`. – mwfearnley Jun 13 '23 at 16:43

Can I use tcpdump to get HTTP requests, response header and response body?

4 Answers4

Linked