3

I need to monitor how long it takes for a certain website to respond when addressed. I would like to sniff the traffic on port 80 but only when there is traffic being exchanged with the targeted site. I have searched SO and it seems like pcapy or scapy is the right tool for the job, but they seem deeper than I need. I have studying the following script:

Network traffic monitor with pcapy in python

and I think I need to change the 

def __handle_packet(self, header, data):
    # method is called for each packet by dispatch call (pcapy)
    self._dispatch_bytes_sum += header.getlen() #header.getlen() #len(data)
    logger.debug("header: (len:{0}, caplen:{1}, ts:{2}), d:{3}".format(header.getlen(), header.getcaplen(), header.getts(), len(data)))
    #self.dumper.dump(header, data)

to somehow only unpack/handle packets that are destined for the target site. Note that this is for a Windows XP machine on a LAN and it is critical that the browser initiate the traffic.

Any pointers appreciated?

Community
  • 1
  • 1
reckoner
  • 2,861
  • 3
  • 33
  • 43
  • [Bro](http://stackoverflow.com/a/11081321/1170277) would be the appropriate tool to use for this scenario. However, it does not work on Windows. – mavam Aug 17 '12 at 21:39
  • to answer my own question, the answer is here: http://pypi.python.org/packages/source/N/NetCube/NetCube-0.3.0.zip. You have to look in the network.py file. – reckoner Sep 05 '12 at 13:15

1 Answers1

3

The problem with scapy is it doesn't handle reassembling TCP streams. Your HTTP that you're looking for is likely to be embedded in a TCP stream. To quote the docs:

Scapy is based on a stimulus/response model. This model does not work well for a TCP stack. On the other hand, quite often, the TCP stream is used as a tube to exchange messages that are stimulus/response-based.

Like you said scapy is more ideal for lower-layer things. You could, for instance, probably track IP packets on DHCP requests. Like many network tools, the complexities and stream-based nature of TCP means once you cross that layer it gets harder to reassemble everything and deal with all the retransmission and what not edge cases and coherently pull data out.

Could you use something like curl or urllib and see how long it takes for the response to come back?

Doug T.
  • 64,223
  • 27
  • 138
  • 202
  • Yes, I'm currently using selenium to do that since it generates traffic from the browser. I was hoping that passively sniffing the traffic that the browser generates would be possible since it is not always possible to configure a corresponding selenium session, but from what you are saying, it looks like that would be quite a challenge. – reckoner Aug 17 '12 at 16:50
  • @reckoner Yeah sniffing TCP is nontrivial. Its stateful. You could have connections setup and stay open for multiple requests, or maybe a new connection would be setup for each HTTP request. You could of course write some code that makes some assumptions that the TCP is always reliable and behaves a specific way that you know only works in the specific case. – Doug T. Aug 17 '12 at 16:54
  • @reckoner scapy's also a little annoying in that in only runs in python 2.6 last I tried to use it. – Doug T. Aug 17 '12 at 16:56