Read last frame from PiCamera Ring Buffer

Question

I am building a motion detector using PiCamera that sends an image to a message queue to have ML detect a person (or not) which then responds back to the client. The motion detection code is mostly lifted from pyimagesearch.com and is working well.

I now want to save video (including the original few frames) once I know a person is detected. Because of the delay in detecting a human (~3 secs) I plan on using a circularIO ring buffer to ensure I capture the initial few seconds of movement. This means I need to change how I loop through frames in detecting motion. Currently I am using:

rawCapture = PiRGBArray(camera, size=tuple(resolution))
camera = PiCamera()
for f in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
    frame = f.array
    # resize, blur, grayscale frame etc.
    # compare to average of last few frames to detect motion

My understanding is this just grabs the Latest frame (ie. not every frame) and keeps on looping until motion is detected. This is what I want as I dont want the process to lag behind real-time by trying to check every frame.

To change this looping logic to leverage the ring buffer I think I can use something like this:

camera = PiCamera()
with camera:
    ringbuf = picamera.PiCameraCircularIO(camera, seconds=60)
    camera.start_recording(ringbuf, format='h264')
        while True:
            # HOW TO GET LATEST FRAME FROM ringbuf???
            # resize, blur, grayscale frame etc.
            # compare to average of last few frames to detect motion

As indicated in the snippet above, how can I get the latest frame as a numpy array from ringbuf to perform by motion detection checking? I have seen examples that get every frame by adding motion_output=detect_function when start_recording is called, but this will check every frame and I dont want the process to lag behind real-time action.

did you see [Advanced Recipes](https://picamera.readthedocs.io/en/release-1.13/recipes2.html) in PiCamer documentation? There are some example with numpy.array but I don't know if they works with video stream. But maybe it uses pure Python code (without C/C++ code) so you could see source code and learn how to do it with video. — furas, Dec 20 '20 at 03:28
in Advanced Recipes in [Custom outputs](https://picamera.readthedocs.io/en/release-1.13/recipes2.html#custom-outputs) I see example which uses `start_recording` with custom class `MyMotionDetector` which converts frame to numpy array. — furas, Dec 20 '20 at 03:32
maybe you should check if `PiCameraCircularIO` is created in pure Python and then you could use it with some modifications. — furas, Dec 20 '20 at 03:34
documentation for [PiCameraCircularIO](https://picamera.readthedocs.io/en/release-1.10/api_streams.html#picamera.streams.PiCameraCircularIO) shows it has variable `.frames` and maybe you can get last frame with `ringbuf.frames[-1]` — furas, Dec 20 '20 at 03:37
Thanks @furas. I just realised that the code I am using now puts each frame into PiRGBArray by using format="bgr" in capture_continuous. But I think for CircularIO to record a rolling chunk of video it needs to be format='h264'. So I would also need a way to extract a frame from h264 and convert to RGB. — Lee Melbourne, Dec 20 '20 at 04:33
The documentation is not very clear on how to use the frames capabilitity and whether you can extract a single frame. If I try to print(ringbuf.frames[-1]) then I get the following error "TypeError: 'PiCameraDequeFrames' object does not support indexing". I also cannot rune a len() on the frames — Lee Melbourne, Dec 20 '20 at 04:37
Good tip regarding advanced recipes. I see that you can quickly switch between formats if you keep to using video port. presumably you just lose a frame or two from the video. This example here might work: https://picamera.readthedocs.io/en/release-1.12/recipes2.html#splitting-to-from-a-circular-stream — Lee Melbourne, Dec 20 '20 at 05:15

score 0 · Answer 1 · answered Dec 20 '20 at 10:31

So I seem to have solved this. Solution is as below. I am guessing I lose a frame every time I switch to the detect_motion function, but that is fine by me. Use of saveAt lets me export the video to file after a fixed number of seconds once motion is detected.

def detect_motion(camera):
    camera.capture(rawCapture, format="bgr", use_video_port=True)
    frame = rawCapture.array
    # resize, blur, grayscale frame etc.
    # compare to average of last few frames to detect motion    


with camera:
    stream = PiCameraCircularIO(camera, seconds=60)
    camera.start_recording(stream, format='h264')
    try:
        while True:
            time.sleep(.3)
            if detect_motion(camera):
                print('Motion detected!')
                saveAt = datetime.now() + timedelta(seconds=30)
            else:
                print('Nothing to see here')
            if saveAt:
                print((saveAt - datetime.now()).seconds)
                if saveAt < datetime.now():
                    write_video(stream)
                    saveAt = None

So I have basically copied the code that is used on their docs site. But the ringbuffer approach using PiCameraCircularIO simply does not work. I get a massive video file containing however long it ran for. — Lee Melbourne, Dec 21 '20 at 09:33

Read last frame from PiCamera Ring Buffer

1 Answers1