Retry a code if runs indefinitely in Python

Question

I am trying to run a code which tries to fetch data from an API:

client.quotes(instrument_tokens=inst_tokens, quote_type="", isIndex=False)

However, sometimes this code keeps running indefinitely without returning any data and I need to interrupt the code using the keyboard and run my code again. I would like to handle this in my code wherein if it takes more than 10 seconds to fetch the data, I retry fetching this data after say, 10 seconds. I have tried using threading, something like

while True:
    thread = threading.Thread(target=run_with_timeout(inst_tokens))
    thread.start()
    thread.join(timeout=10)
    if thread.is_alive():
        print("Execution of quote API took more than 10 seconds. Retrying in 15 seconds...", str(datetime.datetime.now()))
        thread.join()  # Ensure the thread is terminated before retrying
        time.sleep(retry_delay)  # Wait before retrying
        continue  # Restart the loop
    break  # Exit the loop if code ran within time limit

where run_with_timeout is essentially the client.quotes function. Yet this doesn't seem to work and my code still gets stuck running indefinitely. Can anyone please help me with this issue.

Why would you need to use the threading here? You can take start_time and end_time and calculate for 10 seconds, if it exceeds then pause (i.e., add a sleep(10)) and let loop iterate again — 5war00p, Aug 18 '23 at 05:18
If you want to stick with threads, try using ```concurrent.futures```'s ```ThreadPoolExecutor``` and ```TimeoutError```. — Berkay Kaplan, Aug 18 '23 at 05:19
@5war00p since the code runs indefinitely whenever it gets stuck, I don't get any end_time. Thats why I need to check if its been 10 seconds since its running and then retry the code. — Kuljeet Keshav, Aug 18 '23 at 05:21
Thanks @BerkayKaplan! This is my first time using threads. I will look for these functions in the doc — Kuljeet Keshav, Aug 18 '23 at 05:22
If `run_with_timeout` is essentially the `client.quotes` function, then `run_with_timeout(inst_tokens)` is essentially the `client.quotes(inst_tokens)`. You need to do something like `threading.Thread(target=run_with_timeout, args=(inst_tokens,))`. — ken, Aug 18 '23 at 05:30
You need to review the constructor for Thread. Looks like you're giving the output of *run_with_timout* as the target. That's almost certainly **not** what you want (unless it returns a reference to some other function) — DarkKnight, Aug 18 '23 at 05:32

score 1 · Answer 1 · answered Aug 18 '23 at 05:32

1

Maybe you should use requests module for handling api requests. It has a built-in get() which has a timeout parameter which stops an indefinite loop after the given time has elapsed.

answered Aug 18 '23 at 05:32

Nanomachines_Son

11
3

DarkKnight · Answer 2 · 2023-08-18T07:36:00.640

You could use an alarm. Then you wouldn't need a thread.

Something like this:

from time import sleep
from signal import signal, SIGALRM, alarm

TIMEOUT = 2
SLEEP = 5
MAXRETRIES = 5

class Timeout(Exception):
    ...

def run_with_timeout(inst_tokens):
    def _handler(*_):
        raise Timeout

    # don't know what client is so just make something up
    class Client:
        def quotes(self, instrument_tokens, quote_type, isIndex):
            ...

    client = Client()
    # assign a handler for SIGALRM
    handler = signal(SIGALRM, _handler)
    try:
        # set the alarm
        alarm(TIMEOUT)
        return client.quotes(instrument_tokens=inst_tokens, quote_type="", isIndex=False)
    except Timeout:
        # alarm was triggered - i.e., client.quotes() took longer than TIMEOUT seconds
        pass
    finally:
        # make sure the alarm and its handler are reset
        alarm(0)
        signal(SIGALRM, handler)

for _ in range(MAXRETRIES):
    if (result := run_with_timeout('MSFT')) is not None:
        print(result)
        break
    print('Sleeping')
    sleep(SLEEP)
    print('Retrying')
else:
    print('Too many retries')

Thanks for this. I tried this but it seems that SIGALRM doesn't work on my system. It gives import error. It seems it doesn't support Windows systems — Kuljeet Keshav, Aug 22 '23 at 16:42
@KuljeetKeshav As you didn't tag your OS I just assumed you were using a "proper" operating system. You are quite right that SIGALRM is only available on Unix type systems as per the documentation: https://docs.python.org/3/library/signal.html — DarkKnight, Aug 23 '23 at 06:12

Retry a code if runs indefinitely in Python

2 Answers2