This all works locally, but fails in the dev_appserver.py.
I have a global variable to keep track of run status in a Flask app. The variable and all the function below are in the same file.
# Global Variable to keep track of when each run is finished
runcodes = {}
When a new run is started, the run name and code is added to runcodes. run_KAT_thread then starts some longish calculations, while end_run_cleanup polls the value of the run in runcodes. The print outs of runcodes in all these functions gives me the expected value.
@run.route("/run_kat/<string:kat_runname>", methods=["GET"])
@login_required
def kat_run(kat_runname):
# If this is a new run coming from kat_upload
global runcodes
runcodes[kat_runname] = RUN_GOING
print("first runcodes")
print(runcodes)
# This thread runs the KAT executable. It will update runcode[kat_runname] to RUN_END once it finishes
thread_kat = threading.Thread(target=run_KAT_thread, args=(run.kmer_size, run.kat_runname, run.folder, kat_runname))
thread_kat.start()
# This thread cleans up after thread_kat is finished. It needs to be on a different thread in order to keep KAT and the
# app separate (KAT doesn't need to know about databases, apps, or anything).
# It was previously part of the runcode route, but then if the user closed the browser there was no clean up.
thread_end_run_cleanup = threading.Thread(target=end_run_cleanup, args=(current_app._get_current_object(), run))
thread_end_run_cleanup.start()
return render_template("run/in_progress.html", run=run, guest=GUEST)
def run_KAT_thread(kmer_size, kat_project_name, project_path, run_id):
kat = KAT()
global runcodes
print("kat_thread")
print(runcodes)
runcodes[run_id] = kat.run(kmer_size, kat_project_name, project_path)
def end_run_cleanup(app, run):
global runcodes
print("end_run")
print(runcodes)
polling2.poll(lambda: runcodes[run.kat_runname] == RUN_END, step=20, timeout=86400)
print("finished polling")
del runcodes[run.kat_runname]
if run.user == GUEST:
if os.path.exists(run.folder) and os.path.isdir(run.folder):
shutil.rmtree(run.folder)
else:
with app.app_context():
# You need to recapture the run before updating it
current_run = Run.query.filter_by(kat_runname=run.kat_runname).first()
current_run.set_runcode(RUN_END)
db.session.commit()
The kat_run endpoint leads to a in_progress.html. Inside this html there's a javascript function that is fetching the runcode endpoint. In this endpoint, if the run_KAT_thread is finished then I return runcode = RUN_END
@run.route("/runcode/<string:kat_runname>", methods=["GET"])
@login_required
def runcode(kat_runname):
global runcodes
print("runcodes")
print(runcodes)
# Once the subprocess running in thread_kat is finished, the function in thread_end_run_cleanup function will delete
# kat_runname from the runcodes dictionary. Since there will still be some async calls to this route left in the javascript
# queue, we need to check for the presence of kat_runname in order to handle these additional calls.
if kat_runname in runcodes:
data = {"runcode" : runcodes[kat_runname], "next_URL" : url_for("result.get_result", kat_runname=kat_runname)}
else:
data = {"runcode" : RUN_END, "next_URL" : url_for("result.get_result", kat_runname=kat_runname)}
return jsonify(data)
The printout of runcodes in this last function is empty when running with dev_appserver.py.
I realize this is probably not a great design :), it's part of the learning process. At this point I would rather patch it to make it work rather than redesign the whole thing.
I am expecting runcodes to include the kat_runname variable in the runcode endpoint. From what I read multithreading is acceptable when using Google App Engine, so I'm not sure what the problem is.
