1

I have built a plugin-based application where "plugins" (python modules) can be loaded by imp and then scheduled for later execution by APScheduler, I was able to successfully integrate them but I want to implement persistence in case of crashes or application reestarts, so I changed the default memory job store to the SqlAlchemyJobStore, it works quite well the first time you execute the program: tasks are loaded, scheduled, saved at the database and executed at the right time.

Problem is when I try to load the application again I get this traceback:

ERROR:apscheduler.jobstores.default:Unable to restore job "d3e0f0068df54d15986e9b7b6757f665" -- removing it
Traceback (most recent call last):
  File "/home/jesus/.local/lib/python2.7/site-packages/apscheduler/jobstores/sqlalchemy.py", line 126, in _get_jobs
jobs.append(self._reconstitute_job(row.job_state))
  File "/home/jesus/.local/lib/python2.7/site-packages/apscheduler/jobstores/sqlalchemy.py", line 114, in _reconstitute_job
job.__setstate__(job_state)
File "/home/jesus/.local/lib/python2.7/site-packages/apscheduler/job.py", line 228, in __setstate__
self.func = ref_to_obj(self.func_ref)
  File "/home/jesus/.local/lib/python2.7/site-packages/apscheduler/util.py", line 257, in ref_to_obj
raise LookupError('Error resolving reference %s: could not import module' % ref)
LookupError: Error resolving reference __init__:run: could not import module

So it is obvious that there is a problem when attempting to import the function again

Here is my scheduler initialization:

    executors = {'default': ThreadPoolExecutor(5)}
    jobstores = {'default': SQLAlchemyJobStore(url='sqlite:///jobs.sqlite')}
    self.scheduler = BackgroundScheduler(executors = executors,jobstores=jobstores)

I have a "tests" dictionary containing the "plugins" that should be loaded and some parameters, "load_plugin" uses imp to load a plugin by it's name.

for test,parameters in tests.items():
        if test in pluggins:
            module=load_plugin(pluggins[test])
            self.jobs[test]=self.scheduler.add_job(module.run,"interval",seconds=parameters["interval"],name=test)

Any idea about how can I handle reconstituting jobs?

Jesus Gomez
  • 1,460
  • 13
  • 30

1 Answers1

2

Something in the automatic detection of the module name is going wrong. Hard to say what, but the alternative is to manually give it the proper lookup path as a string (e.g. "package.module:function"). If you can do this, you can avoid this problem.

Alex Grönholm
  • 5,563
  • 29
  • 32
  • this fixed my problem indeed, but it wasn't obvious how to implement, for completeness I will edit your answer with the working code. – Jesus Gomez Oct 12 '15 at 15:32
  • yes, it is module error..i create a function with name callback in app.py which is passed to add_job...it works fine...but when i restart app...it cant find the function name...i mean i checked the API while debugging...it gets all methods of app.py in form of dictionary but not callback...why? any help ? – Saad Abdullah Nov 23 '15 at 19:16