24

IDLE is my favorite Python editor. It offers very nice and intuitive Python shell which is extremely useful for unit-testing and debugging, and a neat debugger.

However, code executed under IDLE is insanely slow. By insanely I mean 3 orders of magnitude slow:

bash

time echo "for i in range(10000): print 'x'," | python

Takes 0.052s,

IDLE

import datetime
start=datetime.datetime.now()
for i in range(10000): print 'x',
end=datetime.datetime.now()
print end-start

Takes:

>>> 0:01:44.853951

Which is roughly 2,000 times slower.

Any thoughts, or ideas how to improve this? I guess it has something to do with the debugger in the background, but I'm not really sure.

Adam

Community
  • 1
  • 1
Adam Matan
  • 128,757
  • 147
  • 397
  • 562
  • As answers indicate, slow handling of long lines is a known issue with tcl/tk's line-oriented Text widgets. The result with the recent tcl/tk 8.6.4, distributed with 3.5.0 on Windows, is about the same. – Terry Jan Reedy Oct 11 '15 at 22:16

2 Answers2

31

The problem is the text output not the debugger.

I just tried it on my Q6600 (3GHz overclocked) System and my numbers are even worse. But its easy to see that they are going down the more output text is added.

I tried to run it with

1000 iterations => 7,8 sec 2000 iterations => 28,5 sec 3000 iterations => 70 sec

I did some low level TK stuff in the past and i know that the TkText Widget is keeping the text in a BTree structure. Appending text a character a time is one of the worst ways to do but this seems to be what IDLE is doing. The normal way is to catch more data and append a larger chunk of text.

Amazingly if you write print 'x\n' the output is much faster. 3000 iterations in 7 seconds and your 10000 in 19 sec.

So the problem is definitely with appending single chars to existing lines. The IDLE programmer didn't know how TkText works.

So the advise is to add more newlines into your text or output larger chunks and not only a single 'x' character.

Lothar
  • 12,537
  • 6
  • 72
  • 121
  • +1 for great, comprehensive answer. Do you happen to know if IDLE developers see this as a bug? – Adam Matan Feb 06 '10 at 12:02
  • It's not a bug in IDLE; if it is a bug, then it's in Tkinter's Text widget. – tzot Feb 06 '10 at 19:30
  • 6
    Are there still any IDLE developers? I thought this program is in pure maintenance mode. I personally would call this a bug. – Lothar Feb 06 '10 at 20:59
  • Lothar, I and, I am sure, previous IDLE developers, know well enough how tk Text widgets work. It is definitely line oriented, but handles long lines badly. Such are not typically of real applications. While anyone who both generates text and inserts text into a Text widget can buffer and insert large chunks as appropriate, IDLE does not generate text. It only inserts text generated by programmers, as and when directed. If someone says 'append 1 char to the line', IDLE does that. It would be a bug if it did not. IDLE has no way of knowing how long before there will be another `print`. – Terry Jan Reedy Oct 11 '15 at 22:16
  • Sorry Terry you are wrong with the line orientation. I DID READ THE SOURCE. And i'm very sure you totally overestimate the skills of the IDLE developers regarding TK. – Lothar Oct 23 '15 at 16:05
  • 2
    tzot: it's not a bug but a design limitation - tk as well as many earlier GUI toolkits do immediate update unless otherwise specified (batch operation). Newer toolkits such as WPF and JavaFX switched to a background & independent thread for UI rendering - detect changes in the widget tree and redraw according to your screen refresh rate - which thus nullifies the above issue, but not without introducing overhead of other sorts. In WPF they even added timed event callbacks to address similar problems in application-layer code. – jiping-s Jun 14 '16 at 12:41
11

The problem is in the Tkinter Text widget, and its inefficient management of very long lines, and you create one. You'll notice that, while any part of a very long line is visible, all scrolling is devilishly slow.

tzot
  • 92,761
  • 29
  • 141
  • 204