tqdm: extract time passed + time remaining?

Question

I have been going over the tqdm docs, but no matter where I look, I cannot find a method by which to extract the time passed and estimated time remaining fields (basically the center of the progress bar on each line: 00:00<00:02).

 0%|          | 0/200 [00:00<?, ?it/s]
  4%|▎         | 7/200 [00:00<00:02, 68.64it/s]
  8%|▊         | 16/200 [00:00<00:02, 72.87it/s]
 12%|█▎        | 25/200 [00:00<00:02, 77.15it/s]
 17%|█▋        | 34/200 [00:00<00:02, 79.79it/s]
 22%|██▏       | 43/200 [00:00<00:01, 79.91it/s]
 26%|██▌       | 52/200 [00:00<00:01, 80.23it/s]
 30%|███       | 61/200 [00:00<00:01, 82.13it/s]
....
100%|██████████| 200/200 [00:02<00:00, 81.22it/s]

tqdm works via essentially printing a dynamic progress bar anytime an update occurs, but is there a way to "just" print the 00:01 and 00:02 portions, so I could use them elsewhere in my Python program, such as in automatic stopping code that halts the process if it is taking too long?

casper.dcl · Answer 1 · 2022-12-09T19:56:15.683

21

tqdm objects expose some information via the public property format_dict.

from tqdm import tqdm
    
# for i in tqdm(iterable):
with tqdm(iterable) as t:
    for i in t:
        ...
        elapsed = t.format_dict['elapsed']
        elapsed_str = t.format_interval(elapsed)

Otherwise you could parse str(t).split()

edited Dec 09 '22 at 19:56

answered Dec 06 '19 at 09:51

casper.dcl

13,035
4
31
32

tqdm ```format_dict``` exposes ```elapsed```, but is there any way to retrieve the time remaining, please? – glezo Nov 24 '21 at 11:35
Where should I put the loop? If inside (e.g. if instead of '...' the loop is not moving at all . Essentially how to rewrite "for i in tqdm(range(100)):" so when its finished I can access time elapsed value (ideally also iterations per seconds what does not seems to be included in format_dict) ? – stam Dec 08 '22 at 13:16

score 2 · Answer 2 · answered Dec 15 '21 at 17:15

2

You can get elapsed and remaining time from format_dict and some calculations.

t = tqdm(total=100)
...
elapsed = t.format_dict["elapsed"]
rate = t.format_dict["rate"]
remaining = (t.total - t.n) / rate if rate and t.total else 0  # Seconds*

answered Dec 15 '21 at 17:15

Artem Khodakov

21
2

score 1 · Answer 3 · answered Nov 29 '22 at 07:48

Here's the answer to the time remaining and time elapsed question:

from tqdm import tqdm
from time import sleep
    
with tqdm(total=100, bar_format="{l_bar}{bar} [ time left: {remaining}, time spent: {elapsed}]") as pbar:
        for i in loop:
            pbar.update(1)
            sleep(0.01)

If needed to be worked with or printed elsewhere:

elapsed = pbar.format_dict["elapsed"]
remains = pbar.format_dict["remaining"]

Josh Karpel · Accepted Answer · 2021-04-11T17:17:37.083

Edit: see the library maintainer's answer below. Turns out, it is possible to get this information in the public API.

tqdm does not expose that information as part of its public API, and I don't recommend trying to hack your own into it. Then you would be depending on implementation details of tqdm that might change at any time.

However, that shouldn't stop you from writing your own. It's easy enough to instrument a loop with a timer, and you can then abort the loop if it takes too long. Here's a quick, rough example that still uses tqdm to provide visual feedback:

import time
from tqdm import tqdm


def long_running_function(n, timeout=5):
    start_time = time.time()

    for _ in tqdm(list(range(n))):
        time.sleep(1)  # doing some expensive work...
        elapsed_time = time.time() - start_time
        if elapsed_time > timeout:
            raise TimeoutError("long_running_function took too long!")


long_running_function(100, timeout=10)

If you run this, the function will stop its own execution after 10 seconds by raising an exception. You could catch this exception at the call site and respond to it in whatever way you deem appropriate.

If you want to be clever, you could even factor this out in a tqdm-like wrapper like this:

def timed_loop(iterator, timeout):
    start_time = time.time()
    iterator = iter(iterator)

    while True:
        elapsed_time = time.time() - start_time
        if elapsed_time > timeout:
            raise TimeoutError("long_running_function took too long!")

        try:
            yield next(iterator)
        except StopIteration:
            pass


def long_running_function(n, timeout=5):
    for _ in timed_loop(tqdm(list(range(n))), timeout=timeout):
        time.sleep(0.1)


long_running_function(100, timeout=5)

If there is no way `tqdm` allows this currently (which is a surprise), this is still a good answer for verifying that. Thank you for all the code in implementing `tqdm` in a manual way that allows time checking, which I hadn't thought of myself yet. — Coolio2654, Jun 20 '19 at 00:18
This is the wrong answer. @casper.dcl gave the correct answer as author/maintainer — nurettin, Dec 17 '20 at 12:13
Why is a blatantly wrong answer accepted and the correct and official one not? Does the OP not realize it is possible to change the accepted answer? Either way it seems to the public benefit to do so — Vainmonde De Courtenay, Apr 26 '21 at 16:08

tqdm: extract time passed + time remaining?

4 Answers4