python socket pickled numpy data not showing up

Question

I'm trying to send python-opencv frames over sockets. I'm pickling the data and unpickling but for some reason it's blank or nothing is showing up.

This is my terminal when I run client.py

new message length: b'720       '

It should be streaming the webcam from server but nothing is showing up.

Here is my code for the client and server:

client.py

import socket
import numpy as np
import cv2
import pickle

HEADERSIZE = 10

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((socket.gethostname(), 1232))

while True:
    full_msg = b''
    new_msg = True

    while True:
        msg = s.recv(16)
        if new_msg:
            print(f'new message length: {msg[:HEADERSIZE]}')
            msglen = int(msg[:HEADERSIZE])
            new_msg = False
                                                  
        full_msg += msg

        if len(full_msg)-HEADERSIZE == msglen:
            print('full msg recvd')
            print(full_msg[HEADERSIZE:])

            d = pickle.loads(full_msg[HEADERSIZE:])
            print(d)

            cv2.namedWindow('Webcam', cv2.WINDOW_NORMAL)
            cv2.imshow('Webcam', full_msg[HEADERSIZE:])

            new_msg = True
            full_msg = b''

    print(full_msg)

server.py

import socket
import numpy as np
import cv2
import time
import pickle
from signal import signal, SIGPIPE, SIG_DFL
signal(SIGPIPE, SIG_DFL)

HEADERSIZE = 10

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((socket.gethostname(), 1232))
s.listen(5)

cap = cv2.VideoCapture(0)

while True:
    clientsocket, address = s.accept()
    print(f"Connection from {address} has been established!")

    while True:                                                                   
        ret, frame = cap.read()                                       

        msg = pickle.dumps(frame)
        print(frame)
        msg = bytes(f'{len(frame):<{HEADERSIZE}}', "utf-8") + msg

        clientsocket.send(msg)

I have no idea why nothing is showing up. I don't even know if anything is coming though. Does it have to do with numpy data? I heard that can be tricky.

Liju · Accepted Answer · 2020-09-13T16:02:06.793

When you stream frames of 720 bytes from server, you are actually sending 730 bytes (length 10 bytes + data 720 bytes) per frame continuously, one frame after another.
In client you are reading 16 bytes per recv(). Hence your condition if len(full_msg)-HEADERSIZE == msglen: will never be true, with header size 10, as 730 is not divisible by 16.

So your program is looping indefinitely on while True: in client.

Try below program for client. I tested with dummy data.

client.py

import socket
import numpy as np
import cv2
import pickle

HEADERSIZE = 10

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((socket.gethostname(), 1232))

while True:
    msg_length=int(s.recv(HEADERSIZE))
    full_msg=b''
    while len(full_msg)<msg_length:
        full_msg+=s.recv(msg_length-len(full_msg))
    
    print(full_msg)
    
    d = pickle.loads(full_msg)
    cv2.namedWindow('Webcam', cv2.WINDOW_NORMAL)
    cv2.imshow('Webcam', full_msg)

This solved the problem I asked initially but now I have a new problem. Actually in the time that you replied with your solution I had redone my program but instead I didn't use pickle and did the conversion to bytes my self. I got to the same spot as your solution: `Traceback (most recent call last): File "client.py", line 21, in cv2.imshow('Webcam', full_msg) TypeError: Expected Ptr for argument 'mat'` — throw333, Sep 13 '20 at 20:28

python socket pickled numpy data not showing up

1 Answers1