4

I wrote a simple server and client apps, where I can switch between TCP, DCCP and UDP protocols. The goal was to transfer a file from the one to the other and measure the traffic for each protocol, so I can compare them for different network setups (I know roughly what the result should be, but I need exact numbers/graphs). Anyway after starting both apps on different computers and starting tcpdump I only get in the tcpdump-log the first few MBs (~50MB) from my 4GB file. The apps are written in a standard C/C++ code, which could be found anywhere on the web. What may be the problem or what could I be doing wrong here?

-- Edit

The command line I use is:

tcpdump -s 1500 -w mylog

tcpdump captures then packets only the first ~55 sec. That's the time the client needs to send the file to the socket. Afterwards it stops, even though the server continues receiving and writing the file to the hard drive.

-- Edit2

Source code:

client.cpp
server.cpp
common.hpp
common.cpp

-- Edit final

As many of you pointed out (and as I suspected) there were several misconceptions/bugs in the source code. After I cleaned it up (or almost rewrote it), it works as needed with tcpdump. I will accept the answer from @Laurent Parenteau but only for point 5. as it was the only relevant for the problem. If someone is interested in the correct code, here it is:

Source code edited

client.cpp
server.cpp
common.hpp
common.cpp

stefita
  • 1,785
  • 2
  • 20
  • 35
  • 1
    by default tcpdump only captures the first handful of bytes in each packet. Did you tell it to capture everything, using the `-s snaplen` parameter ? – nos Jul 27 '10 at 09:28
  • @nos, thanks! I tried it out, but still no luck. See my edit. – stefita Jul 27 '10 at 09:44
  • Can you post the tcpdump command used to capture the network traffic? – Laurent Parenteau Jul 27 '10 at 16:09
  • @Laurent Parenteau : tcpdump -s 1500 -w mylog – stefita Jul 28 '10 at 07:42
  • Have you verified that the transfer completed successfully? You can compare the md5sum of the original file and the received file to verify that. – Laurent Parenteau Jul 29 '10 at 15:22
  • @Laurent Parenteau : yes, the file was received 100% on the other side. – stefita Jul 30 '10 at 07:33
  • Where are you performing the capture ? On an intermediate host ? On one of the hosts ? – ereOn Jul 30 '10 at 09:16
  • @ereOn: on both- server and client and both stop logging after the same amount of time. – stefita Jul 30 '10 at 09:29
  • The 'c' tag should be removed since your code is C++, not C. – Laurent Parenteau Jul 30 '10 at 14:12
  • In your the last paragraph you state that the server continues receiving even after the client stop sending. If the client isn't sending anything then what is it receiving? – torak Jul 30 '10 at 14:30
  • @torak: well that's actually my question. The file is growing on the server even though the client app is already finished sending and terminated. As I said, there might be a general knowledge leak on my side. – stefita Jul 30 '10 at 14:36
  • Well it could just be as simple as you are able to complete the transfer accross the network faster than the file is able to written to disk. With a 4 GB transfer and 55 second transfer you are a little over what wikipedia suggests is the "maximal" transfer rate, approximately 70 megabytes a second. See http://en.wikipedia.org/wiki/Hard_disk_drive. However, even taking into account buffering at the socket and file IO levels, its a stretch to think that this explains the ~50 MB written after 55 seconds. – torak Jul 30 '10 at 15:13
  • The code in question has so many problems (I can name at least 3 from a cursory examination) that I strongly suspect the observed behavior is a result of bugs of various varieties. I have run tcpdump for days and it still was faithfully capturing the data I asked it to. My suspicion is that the server is writing data to disk that it didn't actually receive from the network and so there is no network traffic for it. – Omnifarious Aug 03 '10 at 07:54

5 Answers5

7

There are many things wrong in the code.

  1. The file size / transfer size is hardcoded to 4294967295 bytes. So, if the file supplied isn't that many bytes, you'll have problems.
  2. In the sender, you aren't checking if the file read is successful or not. So if the file is smaller than 4294967295 bytes, you won't know it and send junk data (or nothing at all) over the network.
  3. When you use UDP and DDCP, the packets order isn't guarantee, so the data received may be out of order (ie. junk).
  4. When you use UDP, there's no retransmission of lost packet, so some data may never be received.
  5. In the receiver, you aren't check how many bytes you received, you always write MAX_LINE bytes to the file. So even if you receive 0 bytes, you'll still be writing to the file, which is wrong.
  6. When you use UDP, since you're sending in a thigh loop, even if the write() call return the same amount of bytes sent that what you requested, a lot of data will probably be dropped by the network stack or the network interface, since there's no congestion control in place. So, you will need to put some congestion control in place yourself.

And this is just from a quick scan of the code, there is probably more problems in there...

My suggestion is : Try the transfer with TCP, do a md5sum of the file you read/send, and a md5sum of the file you receive/save, and compare the 2 md5sum. Once you have this case working, you can move to testing (still using the md5sum comparison) with UDP and DCCP...

For the tcpdump command, you should change -s 1500 for -s 0, which means unlimited. With that tcpdump command, you can trust it that data not seen by it hasn't been sent/received. Another good thing to do is to compare the tcpdump output of the sender with the receiver. This way you'll know if some packet lost occurred between the two network stacks.

Laurent Parenteau
  • 2,516
  • 20
  • 31
  • 1. and 2. are reasonable, but it's just for test purposes and keeps the source code compact. 3., 4. and 6. are also known, the purpose of the tests is not to implement reliable transfer application, but to test protocol performance (mostly bandwidth). 5. is good point, I'll change that. I already tried the tests with TCP and the file arrives OK and independently of what value I've put for the snaplen tcp won't log behind the ~55sec. – stefita Jul 30 '10 at 15:56
  • And for how long after that the receiver is still receiving data? I am asking that since the data may well be received, but still in the stack's buffers (not yet read by the application). This could explain a small difference between the transfer time on the network, and the time it takes to read everything in the application and write that to the file. – Laurent Parenteau Jul 30 '10 at 16:07
  • it takes longer then a minute to write the rest of the file to the disk. tcpdump captures only handfull of the communication. If I start an ftp transfer than it captures all packets... – stefita Jul 30 '10 at 16:51
  • 1
    Regarding your previous comment "the purpose of the tests is not to implement reliable transfer application, but to test protocol performance (mostly bandwidth)". If you have a lot of packet drop, the performance will seems better than what it really is. Unless, of course, the final application doesn't care about packet drop and a reliable communication... – Laurent Parenteau Jul 30 '10 at 16:58
  • Regarding the md5sum of the sent and received file... Is that file a repeated pattern, or mostly random data? – Laurent Parenteau Jul 30 '10 at 16:58
  • 1
    @stefita - `tcpdump` is capturing all the packets being sent. The numerous bugs in your code are obscuring the fact that it's largely not sending most of the data you think it's sending. The receiver seems to be continuing to 'receive' because bugs in your code cause it to write data to the file even when it's not receiving any. `tcpdump` is functioning perfectly. – Omnifarious Aug 04 '10 at 02:34
3

Do you have x term access? Switch to Wireshark instead and try with that - its free, open source, and probably more widely used than tcpdump today. (It was formerly known as Ethereal.)

Also, do try the following tcpdump options:

  • -xx print the link header and data of the packet as well (does -w write data?)
  • -C specify the max file size explicitly.
  • -U to write packet by packet to the file instead of flushing the buffer.
  • -p dont put the nic in promiscuous mode
  • -O dont use the packet matching optimizer as yours is a new app level protocol.
  • Are you using verbose output in tcpdump? This can make the buffers fill quickly so redirect stdout/err to a file when you run it.

Are these Gigabit ethernet card on both ends?

carlsborg
  • 2,628
  • 19
  • 21
  • Wireshark uses tcpdump/libpcap also, so it wouldn't make any difference. – stefita Aug 02 '10 at 07:53
  • 2
    Well libpcap is a packet capture library and tcpdump and wireshark are two distinct applications that use it. I'm guessing its an application level or configuration problem rather than a library one. – carlsborg Aug 02 '10 at 13:54
  • @stefita: wouldn't or doesn't? Did you try it? – default Aug 04 '10 at 09:28
2

tcpdump is used as a diagnostic and forensics tool by 10s of thousands (at least) programmers and computer security professionals worldwide. When a tool like this seems to be mishandling a very common task the first thing to suspect is the code you wrote, and not the tool.

In this particular case your code has a wide variety of significant errors. In particular, with TCP, your server will continue to write data to the file regardless of whether or not the client is sending any.

This code has race conditions that will result in non-deterministic behavior in some situations, improperly treats '\0' as being a special value in network data, ignores error conditions, and ignores end-of-file conditions. And that's just a brief reading.

In this case I am nearly certain that tcpdump is functioning perfectly and telling you that your application does not do what you think it does.

Omnifarious
  • 54,333
  • 19
  • 131
  • 194
2

"That's the time the client needs to send the file to the socket. Afterwards it stops, even though the server continues receiving and writing the file to the hard drive."

This sound really weird. The socket buffers are way too small to allow this to happen. I really think that your server code only seems to receive data, while the sender actually has already stopped sending data.

IanH
  • 3,968
  • 2
  • 23
  • 26
0

I know this might sound silly, but are you sure it is not a problem of flush() of the file? I.e. the data are still in memory and not yet written to disk (because they do not amount to a sufficient quantity).

Try sync or just wait a bit until you are certain that enough data have been transmitted.

lorenzog
  • 3,483
  • 4
  • 29
  • 50