Keeping Up With Camera Frame Rate in Tkinter GUI

Question

My goal is to display a real time feed from a USB camera in a Tkinter Window. My problem is that I can't seem to update the GUI fast enough to keep up with the frame rate of the camera. I'm interfacing with the camera using the uvclite python wrapper around the libuvc C library. uvclite is a lightweight ctypes wrapper around the underlying C library, so I don't think that piece is my bottleneck. Here is my code:

import tkinter as tk
from PIL import ImageTk, Image
import uvclite
import io
import queue

frame_queue = queue.Queue(maxsize=5)
# frame_queue = queue.LifoQueue(maxsize=5)
user_check = True

def frame_callback(in_frame, user):
    global user_check
    if user_check:
        print("User id: %d" % user)
        user_check = False
    try:
        # Dont block in the callback!
        frame_queue.put(in_frame, block=False)
    except queue.Full:
        print("Dropped frame!")
        pass

def update_img():
    print('getting frame')
    frame = frame_queue.get(block=True, timeout=None)
    img = ImageTk.PhotoImage(Image.open(io.BytesIO(frame.data)))
    panel.configure(image=img)
    panel.image = img
    print("image updated!")
    frame_queue.task_done()
    window.after(1, update_img)


if __name__ == "__main__":

    with uvclite.UVCContext() as context:
        cap_dev = context.find_device()
        cap_dev.set_callback(frame_callback, 12345)
        cap_dev.open()
        cap_dev.start_streaming()

        window = tk.Tk()
        window.title("Join")
        window.geometry("300x300")
        window.configure(background="grey")

        frame = frame_queue.get(block=True, timeout=None)
        # Creates a Tkinter-compatible photo image, which can be used everywhere Tkinter expects an image object.
        img = ImageTk.PhotoImage(Image.open(io.BytesIO(frame.data)))
        panel = tk.Label(window, image=img)
        frame_queue.task_done()
        panel.pack(side="bottom", fill="both", expand="yes")
        window.after(1, update_img)
        window.mainloop()

        print("Exiting...")
        cap_dev.stop_streaming()
        print("Closing..")
        cap_dev.close()
        print("Clear Context")

Each frame is a complete JPEG image, stored in a bytearray. The frame_callback function gets call for every frame generated by the camera. I see "Dropped frame!" printed quite frequently, meaning my GUI code isn't pulling frames off of the queue fast enough and frame_callback encounters the queue.Full exception when trying to put new frames onto the queue. I've tried playing with the delay on the window.after scheduled function (first integer argument, units of milliseconds), but haven't had much luck.

So my question is: What can I do to optimize my GUI code to pull frames off of the queue faster? Am I missing something obvious?

Thanks!

Read about [`Image.frombytes(...`](https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.frombytes). I feel your `frame_queue` can't deliver within **1 millisecond**. — stovfl, Oct 23 '19 at 15:32
What is the frame rate of the camera? I doubt it's 1000 frames per second, which is how often you're attempting to update the image. — Bryan Oakley, Oct 23 '19 at 15:33
I found a better way (inspired by @stovfl 's comment). I'm going to post it as an answer now. — Mr. Frobenius, Oct 23 '19 at 15:45

score 0 · Answer 1 · answered Oct 23 '19 at 15:50

I'm posting this as an answer inspired by @stovfl's comment on the question, but I'm still curious to see how other people would approach this.

His comment pointed out that my frame_queue probably cannot deliver frames when calling frame_queue.get() within 1 millisecond, so I just removed the queue from the GUI update code entirely. Instead, I call the GUI updating code from the callback directly. Here is the new code:

import tkinter as tk
from PIL import ImageTk, Image
import uvclite
import io

user_check = True

def frame_callback(in_frame, user):
    global user_check
    if user_check:
        print("User id: %d" % user)
        user_check = False
    img = ImageTk.PhotoImage(Image.open(io.BytesIO(in_frame.data)))
    panel.configure(image=img)
    panel.image = img

if __name__ == "__main__":

    with uvclite.UVCContext() as context:
        cap_dev = context.find_device()
        cap_dev.set_callback(frame_callback, 12345)
        cap_dev.open()
        cap_dev.start_streaming()

        window = tk.Tk()
        window.title("Join")
        window.geometry("300x300")
        window.configure(background="grey")
        panel = tk.Label(window)
        panel.pack(side="bottom", fill="both", expand="yes")

        window.mainloop()
        print("Exiting...")
        cap_dev.stop_streaming()
        print("Closing..")
        cap_dev.close()
        print("Clear Context")

This works quite well, and the GUI is very responsive and captures motion in real time. I'm not going to mark this as an answer just yet, I would like to see what other people come up with first.

Keeping Up With Camera Frame Rate in Tkinter GUI

1 Answers1

Linked