Why does Tkinter text widget "lags" updates for increased window heights?

Question

I'm currently implementing a sample UI for neovim, and decided to use Tkinter/python due to the popularity/simplicity of the platforms. The problem I'm having is that tkinter seems to "stack" UI updates when the window height crosses a certain threshold.

Here is a video that shows the problem.

The right window is a terminal emulator running neovim, and the left window is the Tkinter UI program connected to it. The idea is that the tkinter UI should mirror neovim terminal screen, including dimensions. In this video never take focus away from the terminal window, so the only events Tk has to process come from the connection to neovim(virtual 'nvim' events which describe screen updates)

The first part of the video shows that everything works nicely when the window height is small, but starts to lag updates when I increase the height.

Here is the code for the Tkinter program. While neovim API is very new and still in heavy development(the code may not make sense to some readers), I think the problem I'm trying to solve is close to implementing terminal emulator(using Tk text widget): it must handle large bursts of formatted text updates efficiently.

I'm very inexperienced in GUI programming. Is Tkinter a wise choice for this task? If yes, then could someone give me a hint of what I'm doing wrong?

To explain a bit what's happening: Neovim API is thread-safe, and the vim.next_event() method blocks(without busy waiting, it uses a libuv event loop underneath) until an event is received.

When the the vim.next_event() call returns, it will notify Tkinter thread using generate_event, which will do the actual event processing(it also buffers events between redraw:start and redraw:stop to optimize screen updates).

So there are actually two event loops running in parallel, with the background event loop feeding the Tkinter event loop in a thread-safe way(the generate_event method is one of the few that can be called from other threads)

Answer 1

I would double check that it is, in fact, Tkinter that's the hold up. The way I would do this is by simply writing out to the terminal when you get an event.

But now that I take a closer look this could be your problem:

    t = Thread(target=get_nvim_events, args=(self.nvim_events,
                                             self.vim,
                                             self.root,))

Threads do not play nicely with event loops - of which Tkinter already has one. I'm not sure if the neovim api is setup to use callbacks, but that's typically how you want to propagate changes.

Since you say you're unfamiliar with GUI programming I'm going to assume that you're unfamiliar with the idea of event loops. Basically, imagine you have some code that looks like this:

while True:
    if something_to_do:
        do_it_now()

Obviously that's a busy loop and will burn your CPU, so typically your event loop is going to block or setup callbacks with the OS, which allows it to relinquish the CPU and when something interesting happens the OS will say something like, "Someone clicked here," or "Someone pressed a key," or "Hey, you told me to wake you up now!"

So your job as a GUI developer is to jack into that event loop. You don't really care when something happens - you just want to respond to it. With Tkinter you can do that with the .after methods see "non-event callbacks". What might be a good choice is the .after_idle method:

Registers a callback that is called when the system is idle. The callback will be called there are no more events to process in the mainloop. The callback is only called once for each call to after_idle.

This means you won't block a key press or a mouse click, and it will only run once Tkinter has finished processing its other stuff (e.g. drawing, calling callbacks, etc.)

I expect what might be happening is that your thread and the mainloop are having issues (possibly thanks to the GIL). I looked around but didn't see anything immediately obvious, but what you want to do is something to the effect of:

def do_something(arg):
    # do something with `arg` here


def event_happened(event_args): #whatever args the event generates
    root.after_idle(lambda: do_something(event_args))

vim.bind("did_something", event_happened)

Of course, it's also possible that you can just bypass the event loop altogether and have the event do what you want.

Answer 2

I have also observed similar issues with Tkinter widgets and have found that it seems to be as a result of issues with Tkinter reducing its performance. Without and overhaul of the Tkinter it seems this problem cannot be resolved.