0

I have a python program running on an embedded Linux board (like a raspberry pi), and another python program running on my PC.

I am trying to use the imageio library as a way of capturing frames from the webcam (e.g., pi camera) of the linux board, send these to my PC via a TCP connection, and then read the frames as numpy arrays on my PC, all with the lowest latency.

(Note: my previous approach was using opencv to compress frames into low-quality jpeg before sending them through TCP on the board and had a latency of ~0.25-0.5s on my local network, but I think it is suboptimal, and I do not want to use opencv).

Thus far I have been doing something like the following, but I think my approach is dumb (and it has a latency > 1s):

# Linux board:

for idx, frame in enumerate(iio.imiter("<video0>", size=(resolution[0], resolution[1]), fps=fps)):
    data = pkl.dumps(frame, 0)
    framelen = len(data)
    connection.write(struct.pack('<L', framelen))
    connection.flush()
    connection.write(data)

# PC:

 while True:
    framelen = struct.unpack('<L', connection.read(struct.calcsize('<L')))[0]
    if not framelen:
        break

    data = connection.read(framelen)  # raw undecoded pickle bytestring
    frame = pkl.loads(data, fix_imports=True, encoding="bytes")

Is there a way of doing something more clever/efficient, like using imageio for sending the encoded/compressed video at the desired resolution and framerate from the board through my python socket and decompressing them on my PC, please?

Thanks.

Yann Bouteiller
  • 47
  • 7
  • What is `connection`? It can't be a socket because python sockets don't have a `write` method. – President James K. Polk Dec 10 '22 at 00:54
  • Maybe consider using the software and hardware specifically developed for the RasPi and its camera... https://picamera.readthedocs.io/en/release-1.13/index.html – Mark Setchell Dec 10 '22 at 10:56
  • Mark Setchell actually I don't want to, because the thing needs to work on Coral boards as well. – Yann Bouteiller Dec 10 '22 at 21:24
  • No problem. But such things should be clarified in your question. – Mark Setchell Dec 10 '22 at 21:31
  • @PresidentJamesK.Polk it is a socket.socket().makefile('wb') – Yann Bouteiller Dec 10 '22 at 21:33
  • @MarkSetchell true, I have edited the part that made it sound it was specifically about the raspi. – Yann Bouteiller Dec 10 '22 at 21:37
  • At the time of this writing, ImageIO only supports reading from HTTP-based streams (e.g. DASH video). The more tinker-friendly protocols like UDP or RTP are in the backlog, but require a large refactor of some internals so no ETA for now. Similarly, there is currently no write-support for remote resources/streams. The efficient way to send video is to use FFMPEG on the host to stream/encode the webcam frames and to then bind to the endpoint using ImageIO (if HTTP-based) or pyav (otherwise) to read and decode the incoming stream. – FirefoxMetzger Dec 15 '22 at 08:36

0 Answers0