0

I'm using the following piece of python code to capture traffic and dump it to a .pcap file:

from pcapy import open_live

p = open_live("eth0", 65535, 1, 0)
dumper = p.dump_open("./test.pcap")
while capturing:
    (header, packet) = p.next()
    dumper.dump(header, packet)

I'm actually running this in a thread, where capturing is a threading.Event() that is set to False when I want to stop the capture (so it exits the loop and returns cleanly).

However, when I try to open the test.pcap with wireshark, I get this message:

The capture file appears to have been cut short in the middle of a packet.

I sometimes see that there are a couple of packets missing at the end of the test.pcap (I can debug that because I'm writing the intercepted packets in a .csv file). But, besides that, I think the pcap file is fine. This message is a bit annoying, though. I thought that it could be that I need to include some magic number to make wireshark believe it's a wireshark capture or something like that. I found other questions of people getting this message because they don't close the capture cleanly (but as I said, I do (or do I need to explicitly close the file descriptor open by pcapy? I couldn't find a method in the pcapy API that closes it). Also, I'm not capturing with wireshark, so it might be a different problem).

Does anybody know what is the message due to? Or, does anybody know how can I debug and find the cause that makes wireshark pop this message?

EDIT

Pcapy source code that closes dumper here.

Community
  • 1
  • 1
synack
  • 1,699
  • 3
  • 24
  • 50

2 Answers2

1

I found other questions of people getting this message because they don't close the capture cleanly

What's not being closed cleanly is the handle for the dumper.

or do I need to explicitly close the file descriptor open by pcapy?

You need to close dumper.

I couldn't find a method in the pcapy API that closes it

There isn't one. From a quick look at the pcapy source code, it's closed implicitly when dumper gets released. That should be happening if your Python program exits cleanly.

If it's not exiting cleanly (for example, if it's being killed by control-C without it, or the Python interpreter, catching the signal and cleanly exiting), you need to fix that.

If it is exiting cleanly, there's probably a bug somewhere in pcapy; report that as a bug to the pcapy developers.

(pcap_dumper_ts use the "standard I/O library" C routines, which means they do buffered output, and data doesn't get written out to the file immediately with every dump call; closing the dumper causes data not yet written to the file to be written out. That should be done when the C program - which would be the Python interpreter - exits, if it's not done before that.)

  • Thanks for your answer. I'm doing several captures, so my program does not exit after the thread where I run the code above runs. When I set the event to False, the thread should return and that should close the dumper, right? – synack Apr 11 '15 at 17:46
1

pcapy tries to mimic the API of the pcap library. Unfortunately it is missing an important function: pcap_dump_flush. By default pcap buffers writes and will only write the buffer to disk if it it contains enough data or if the dump file gets closed. With pcap_dump_flush on can trigger the flushing of the buffer. Since this function is not available from pcapy your only choice is to close the file before reading it from another application.

Steffen Ullrich
  • 114,247
  • 10
  • 131
  • 172
  • Thanks for the answer. So, combining your answer with @Guy Harris', my first shot to close the file is to try to release `dumper` by doing something like `del dumper`? Is that right? – synack Apr 11 '15 at 17:48
  • `dumper = None` should work too. This will free the object and from a look at [the code](https://github.com/CoreSecurity/pcapy/blob/master/pcapdumper.cc) it will call `pcap_dump_close` which is what you need. – Steffen Ullrich Apr 11 '15 at 17:54
  • Yes, that was enough. Thank you. – synack Apr 11 '15 at 18:03
  • Yes - automatic garbage collection may be sufficient if freeing an object has no useful side-effects, so that it doesn't matter when it's done. Unfortunately, that's not true for I/O handle objects; if, for example, buffers get flushed when the object gets released, deferring that flushing until garbage collection frees up the object defers writes to the file. Explicit release calls may be necessary in those cases. –  Apr 11 '15 at 18:07
  • @GuyHarris: yes, one is probably lucky if the objects gets freed immediately this way. But I don't know of anything better: I think it is a design error that close of dump file is only done on garbage collection, apart from the missing functionality for flush. This design will work with a GC based on reference counting (like in Perl) but not with a GC which gets called whenever it thinks it makes sense. – Steffen Ullrich Apr 11 '15 at 18:52
  • "I think it is a design error that close of dump file is only done on garbage collection" "close" for a dump file is "close and destroy"; when `pcap_dump_close()` returns, the `pcap_dumper_t` is no longer valid. I'm not sure what, in a language with garbage collection, the appropriate way to represent that is. Marking an object as reference-counted and having it released as soon as the reference count goes to zero, rather than deferring reclamation until memory is needed? –  Apr 11 '15 at 20:25
  • @GuyHarris: I think the correct way should be for the user to close (and flush) the file explicitly, instead of relying on GC. – Steffen Ullrich Apr 11 '15 at 20:42
  • Any language where you can have objects that affect "the outside world", even if that just means writing to a file, probably needs a way to explicitly terminate objects' connection with the outside world. Should those objects be allowed to exist but return "I'm disconnected", or should there be a way to arrange that the object be disconnected and destroyed? –  Apr 12 '15 at 03:57
  • I think it should be handled like sockets. There is a method to close the socket because there are times when the user needs to close the socket now and not when the GC runs. Any future operations on the socket will fail after it was closed. – Steffen Ullrich Apr 12 '15 at 07:06