I would like to use picamera2 together with Gstreamer, using a Raspberry Pi Zero 2W and a Picamera Module V2, having in mind that:
During the operation, I will need to change the resolution, crop the image, control exposure, take pictures, etc. It would be nice if those changes are as fast and smooth as possible (picamera2 definitely helps with this!).
Video coded with H264.
Using Gstreamer, mux KLVs into a MPEG transport stream, which contain different information regarding the position of the camera, timestamp, field of view, etc. This data changes over time and is fed into the stream using the element appsrc through the need_data signal.
The final stream is packetized using RTP and sent via UDP.
I have been using the Gstreamer python bindings Gst, together with the raspicam plugin rpicamsrc. Since there is no such a plugin for picamera2, I was wondering if there is a way of doing something similar.
Things I have tried:
Stream the camera video, already H264 compressed, similarly to the example capture_stream_udp.py, to the loopback address. Take that stream into the Gst pipeline, inject KLVs, packetize with RTP, and send it via UDP. It works "fine", but the video delay is higher and there is a reduction in the FPS (the video is clearly less smooth).
Capturing the video frames into a numpy array and passing them to the Gst pipeline, using appsrc. For a Pi Zero 2, 720p, 40 fps, this is not viable. Does anyone know how it could be done? Any clue?
I was wondering that maybe it could be done using file descriptors (somehow using FileOutput at the picamera2 side, and fdsrc at the Gst side??).
Thanks is advance
