Python Sockets - Data is being sent way too fast

Question

I'm using python sockets to send data across however whenever I'm sending data to the client, it seems to miss my data unless I'm debugging (which allows me to pause execution when needed).

Server snippet:

def send_file(client_socket: socket):
    with open('client.py', 'rb') as file:
        while True:
            read_data = file.read()
            client_socket.sendall(read_data)
            if not read_data:
                client_socket.sendall('End'.encode())
                break
    print('Finished')

The server reports that it has finished and sent the 'end' message, but my client seems to be hanging on listening for too long, even though I thought adding a end message would help. Client Snippet:

    with open('test.txt', 'wb') as file:
        while True:
            received_bytes = sock.recv(BUFFER_SIZE)
            if received_bytes == b'End':
                break
            file.write(received_bytes)
        # TODO: Restart client program

What am I doing wrong here?

Daweo · Answer 1 · 2022-01-14T08:49:18.010

received_bytes = sock.recv(BUFFER_SIZE)

Does read BUFFER_SIZE or less bytes, depending on BUFFER_SIZE and message you send End might become part of received_bytes rather than whole received_bytes or be cut between subsequent received_bytes. Consider following example, let say your message is HELLOWORLD thus you do send HELLOWORLDEnd and BUFFER_SIZE equals 3 therefore received_bytes are subsequently

HEL
LOW
ORL
DEn
d

Thus last is clearly not End. You do not need special way of informing about end, see part of socket Example from docs:

while True:
    data = conn.recv(1024)
    if not data: break
    conn.sendall(data)

In your case this means not sending End and replacing

if received_bytes == b'End':

using

if not received_bytes:

EDIT after comment server cannot send an empty message and so the client is still listening for data even though it has well sufficiently been sent

If you must use end of message marker AT ANY PRICE, then consider using single byte for that purpose, which never appears in your message. For example if you elect \x00 for this purpose, you might then do

    received_bytes = sock.recv(BUFFER_SIZE)
    file.write(received_bytes.rstrip(b'\x00'))
    if received_bytes.endswith(b'\x00'):
        break

.endswith and .rstrip methods work for bytes same way like for strs. Thus this code does write received_bytes sans trailing \x00 (note that .rstrip does not modify received_bytes) then if received_bytes endswith said byte break.

Buffer size is the same for both, and when I use if not received, the client hangs waiting for another buffer of data to come through. And the end message works when I'm debugging (aka. Full control of time in my program) — Joy Singh, Jan 13 '22 at 21:10
@JoySingh Right, because with the delays due to debugging, the final chunk of data from the file gets sent as a distinct packet, and the 'End' message gets sent in a packet of its own - which is the only case in which your current code can detect it. — jasonharper, Jan 13 '22 at 21:53
@jasonharper Thank you for the explanation. I'm wondering what alternatives exist since in my situation the server cannot send an empty message and so the client is still listening for data even though it has well sufficiently been sent — Joy Singh, Jan 14 '22 at 03:34
@JoySingh I edited my answer to include possible end of message implementation — Daweo, Jan 14 '22 at 08:49

Python Sockets - Data is being sent way too fast

1 Answers1