0

I would like to call tcpdump in a subprocess in my code. The arguments to tcpdump are such that it keeps writing to 5 files cyclically, rotating whenever the file size reaches 1 MB. Following is the code:

tcpdump tcp -w test -C 1 -W 5

Now I would like to read each file one by one in a thread as soon as it is written completely (1 MB) in a cycle. However, I have no way of informing the thread that the file is ready to read, as subprocess module doesn't seem to allow anything more than status/output of the process to be returned. tcpdump doesn't show any output either about successfully writing to a file.

How do I go about solving this problem?

swordfish
  • 393
  • 1
  • 3
  • 5
  • the question seems unrelated to `subprocess` module. Why don't you process the output as soon as it is available? (you can copy it to files too). – jfs Jan 08 '16 at 09:47
  • For optimisation purposes. The processing required takes a lot of power. So it is done periodically from the files created by tcpdump. – swordfish Jan 08 '16 at 11:26
  • I don't see how reading from a file instead of a pipe can improve performance here – jfs Jan 08 '16 at 11:51
  • Processing is done on each packet as it is dumped to MySQL table. Given that thousands of packets are captured in every 10 seconds with high network traffic, writing them to a file and then processing them later (when may be network activity is less) distributes the load. – swordfish Jan 08 '16 at 11:58
  • you could use 1M in memory buffer to smooth the spikes. Try to pass `bufsize=1<<20` to `Popen()` and see what happens. – jfs Jan 08 '16 at 12:10
  • It's not about having increased memory to store. The SQL connection can drop and several things can happen and we don't want to lose the captured packets by processing them on the fly. – swordfish Jan 08 '16 at 12:41

0 Answers0