Receiver loses the same last bytes of file lost sent via TCP socket in python

Question

I send a .txt file(about 87 kbyte size) from a client to a server over TCP with the following code(python):

Client:

                        f = open(filename, 'r')
                        while 1:
                            data = f.read(1024)
                            if not data:
                                data='*Endoffile*!'
                                con.send('%1024s' %data)
                                f.close()
                                break
                            else:
                                con.send('%1024s' %data)

Server:

f = open(filename,'w')
while 1:
    data = c.recv(1045)
    if data=='%1024s' %'*Endoffile*!':    
        f.close()
        break
    else:
        f.write(data)

The problem is that the file received is always cut at the same point(it stops always at the same word when i open it), about 1,6 kbyte smaller than the size of the complete file. Does anyone have a suggestion of what to do?

What exactly is missing? Instead of reading and writing files, generate a pattern on the sender side and verify the pattern on the receiver side. Also, what Python version are you using? — Ulrich Eckhardt, Jun 12 '14 at 18:04
I use fixed size data to send and the string '*Endoffile*!' to recognize that there is no data to receive anymore. — Orion Papadakis, Jun 12 '14 at 18:12
"1045 is the size in bytes for 1024 characters in python" - that's wrong, the size depends on the actual characters and the used encoding. Again, write a byte-pattern and verify it on the other side to rule out failure to read or write a file (divide and conquer strategy). Also, unless you have to implement an existing protocol for file transfer, I would suggest that you use e.g. zeromq, which takes away a lot of the low-level networking complexity. — Ulrich Eckhardt, Jun 12 '14 at 18:32

score 2 · Answer 1 · answered Jun 12 '14 at 23:16

Solved by these changes in the server side code:

        while 1:
          f = open(filename,'wb')

          data=c.recv(1024)

          while ('*Endoffile*!' in data) == False:
            f.write(data)
            data=c.recv(1024)

          print data
          data=data.replace('*Endoffile*!', '')
          f.write(data)
          f.close()

score 0 · Answer 2 · edited May 23 '17 at 11:57

0

Basically, the end-of-file check is broken, so the server never breaks out of the loop. The file never gets closed and recently-written data does not get flushed to disk, resulting in a truncated file.

The reason the EOF check is broken is that TCP is a stream-oriented protocol and does not preserve message boundaries. In other words, a single recv() does not map to a single send().

Your code, on the other hand, expects the '%1024s' %'*Endoffile*!' to be received in one complete recv().

For further discussion, see Difference between message-oriented protocols and stream-oriented protocols

edited May 23 '17 at 11:57

Community

1
1

answered Jun 12 '14 at 18:10

NPE

486,780
108
951
1,012

I replaced: data = c.recv(1045) with: buff='' while len(buff) < 1045: buff += c.recv( 1045 - len(buff) ) data = buff It happens exactly the same – Orion Papadakis Jun 12 '14 at 18:27

Receiver loses the same last bytes of file lost sent via TCP socket in python

2 Answers2