0

I have a small script that polls a database to look for status of certain jobs. I decided to use APScheduler to handle the looping call. I created a decorator to timeout a function if taking too long. The issue I am having here is that the decorator is inside a class and even though I create two instances of the class, inside two different functions, they always have the same start_time. I thought maybe if I move the decorator inside of my class and initialize the start_time in the init call it would update the start_time per instance of the class. When I moved the decorator insdie of the class and assigned self.start_time = datetime.now() the start time updates on each call of the class and thus will never time out. The example of the decorator inside of the class is also below.

def timeout(start, min_to_wait):
    def decorator(func):
        def _handle_timeout():
            scheduler.shutdown(wait=False)
        @wraps(func)
        def wrapper(*args, **kwargs):
            expire = start + timedelta(minutes = min_to_wait)
            now = datetime.now()
            if now > expire:
                _handle_timeout()
            return func(*args, **kwargs)
        return wrapper
    return decorator

class Job(object):
    def __init__(self, name, run_id, results):
        self.name = name
        self.run_id = object_id
        self.results = results
        self.parcel_id= None
        self.status = None

    start_time = datetime.now()

    @timeout(start_time, config.WAIT_TIME)
    def wait_for_results(self):
        if self.results:
            self.pack_id = self.results[0].get('parcel_id')
            self.status = self.results[0].get('status')
            return self.results[0]
        else:
            return False

    @timeout(start_time, config.WORK_TIME)
    def is_done(self):
        status = self.results[0].get('status')
        status_map = {'done': True,
                      'failed': FailedError,
                      'lost': LostError}

        def _get_or_throw(s, map_obj):
            value = map_obj.get(s)
            if s in ['failed', 'lost']:
                raise value(s)
            else:
                self.status = s
                return s

        return _get_or_throw(status, status_map)


def job_1(mssql, postgres, runid):
    res = get_results(mssql, config.MSSQL, first_query_to_call)
    first_job= Job('first_job', runid, res)
    step_two = pack_job.wait_for_results()

    if step_two:
        try:
            logger.info(first_job)
            if first_job.is_done() == 'done':
                scheduler.remove_job('first_job')
                scheduler.add_job(lambda: job_two(mssql,
                    postgres, first_job.object_id, runid), 'interval', seconds=config.POLL_RATE, id='second_job')

        except LostError as e:
            logger.error(e, exc_info=True)
            scheduler.shutdown(wait=False)
        except FailedError as e:
            logger.error(e, exc_info=True)
            scheduler.shutdown(wait=False)


def job_two(mssql, postgres, object_id, runid):
    res = get_results(mssql, config.MSSQL, some_other_query_to_run, object_id)
    second_job= Job('second_job', runid, res)
    step_two = second_job.wait_for_results()

    if step_two:
        try:
            logger.info(second_job)
            if second_job.is_done() == 'done':
                scheduler.remove_job('second_job')

        except LostError as e:
            logger.error(e, exc_info=True)
            scheduler.shutdown(wait=False)
        except FailedError as e:
            logger.error(e, exc_info=True)
            scheduler.shutdown(wait=False)

if __name__ == '__main__':
    runid = sys.argv[1:]

    if runid:
        runid = runid[0]

    scheduler = BlockingScheduler()
    run_job = scheduler.add_job(lambda: job_one(pymssql, psycopg2, runid), 'interval', seconds=config.POLL_RATE, id='first_job')

attempt to move decorator inside class:

class Job(object):
    def __init__(self, name, run_id, results):
        self.name = name
        self.run_id = run_id
        self.results = results
        self.pack_id = None
        self.status = None
        self.start_time = datetime.now()


    def timeout(min_to_wait):
        def decorator(func):
            def _handle_timeout():
                scheduler.shutdown(wait=False)
            @wraps(func)
            def wrapper(self, *args, **kwargs):                
                print '**'
                print self.start_time
                print ''
                expire = self.start_time + timedelta(minutes = min_to_wait)
                now = datetime.now()
                if now > expire:
                    _handle_timeout()
                return func(self, *args, **kwargs)
            return wrapper
        return decorator

here is an example output from when I use the above decorator.

**
self start time: 2014-10-28 08:57:11.947026 

**
self start time: 2014-10-28 08:57:16.976828 

**
self start time: 2014-10-28 08:57:21.989064 

the start_time needs to stay the same or else I can't timeout the function.

Ptrkcon
  • 1,455
  • 3
  • 16
  • 32

1 Answers1

0

In the first exemple, your start time is initialised when the class statement is executed, which in your case is when the module is first imported in the interpreter.

In the second exemple, the start time is initialized when the class is instanciated. It should not change from one method call to another for a same Job instance. Of course if you keep on creating new instances, the start time will be different for each instance.

Now you didn't post the code using your Job class, so it's hard to tell what the right solution would be.

bruno desthuilliers
  • 75,974
  • 6
  • 88
  • 118
  • both job_1 and job_two functions use instances of Job(). It makes new instances of the class each time since APScheduler calls either job_1 or job_two function every 'x' seconds. I see this in the print statements and the start_time increments by the 'x' seconds that I have configured. – Ptrkcon Oct 28 '14 at 13:53