3

So I'm trying to write a Flask app that has a block of code running once every ten minutes, pulling down and processing a file. The way I currently trigger that block of code to run is a loop that sees if the delta between the current time and the last time the block of code ran is greater than ten minutes. Multiprocessing is being used to run the loop concurrently with what Flask is doing, all set up in the if __name__ == "__main__": portion of my Flask app.

Unfortunately, since the app is being run using Gunicorn (using Heroku to run the app), if __name__ == "__main__": doesn't resolve as true and the loop doesn't even start. Setting up Multiprocess in something like if True: only throws errors and doing if __name__ =! "__main__": throws no errors but locks Flask up in the loop, meaning that Multiprocess isn't really doing what it's supposed to. Really curious as to why this is, but my larger question is if there is a better way to run my block of code once every ten minutes where I can still easily pass the results of the code to Flask or get Multiprocess to run correctly and would you mind elaborating on either? Something that's built into Flask, maybe similar to @app.before_first_request, where an external trigger isn't required would be nice. I feel as if I've overlooked something really simple and over complicated what I'm doing as a result.

A simplified version of my code is as follows and works when not being handled by Gunicorn on my local machine:

import time
from flask import Flask, request, jsonify
from multiprocessing import Process, Value

app = Flask(__name__)

@app.route('/')
def webhook:
   respond_to_requests()

def record_loop(loop_on):
   while True:
      if loop_on.value == True:
         check_time_and_run_code()
   time.sleep(1)

if __name__ == "__main__":
   recording_on = Value('b', True)
   p = Process(target=record_loop, args=(recording_on,))
   p.start()
   app.run(debug=True, use_reloader=False)
   p.join()

The Procfile I'm using on Heroku to run this app with Gunicorn contains web: gunicorn app:app --log-file=-

Thanks in advance for the help! :D

Boyd Christiansen
  • 83
  • 2
  • 8
  • There doesn't seem to be any reason to run this in the same process as the web app. Run it separately and manage it by something like cron; gunicorn doesn't need to be involved at all. – Daniel Roseman Feb 08 '18 at 07:44

1 Answers1

0

You seem to doing a background task in a web process. Web processes are meant to handle incoming requests. Background tasks should be run separately.

You can use Heroku Scheduler or a custom clock process to run background tasks at regular intervals.

Aditya Bansal
  • 16
  • 2