1

I've read the Heroku page about custom clock processes where there's a web dyno and a clock dyno that runs certain jobs independently with an already-set schedule. What I need is to be able to schedule one-time jobs from the web dyno, i.e. a certain python object from that dyno must be accessible, and I'm not really clear on how these details could be implemented.

Currently, I have a worker dyno with several threads, where one of those threads is the one that does the jobs. This thread is started from the beginning if there's pending jobs. Otherwise, it's started by the main thread when there's a request for a job. From there on, the thread sleeps until it's time to do the job. This waiting can be interrupted by a new request from the main thread and it'll have to check when the next job will be. If there are no pending jobs, the thread ends its execution. The problem is that this approach is already consuming all my free-dyno hours and it doesn't seem a good way of doing things.

I've read this solution where APScheduler's add_job is called from a web dyno, but I don't think that scheduler would even work when the web dyno is asleep, right? How could this be done?

I hope I explained myself well enough.

Javain
  • 11
  • 2
  • Check this https://stackoverflow.com/a/64478590/9095551, it explains how to start a thread from a Web request (one off task), maybe it helps – Beppe C Oct 22 '20 at 18:58
  • That would be what I have now, more or less, some kind of trigger that prepares a thread to do something in a dynamic time frame. But the job may need to wait several hours to start, that's why I think I'm gonna need a scheduler. – Javain Oct 23 '20 at 07:22
  • Yes, my example start the thread right away. You could add a delay, but if you want to control when it runs the best option is a scheduler – Beppe C Oct 23 '20 at 07:34
  • Ok, thank you for reassuring about needing a scheduler. I'm not really sure about how all this works. Do you think a solution like the one I linked in the question would work? I may have been wrong. – Javain Oct 23 '20 at 07:45
  • Generally speaking, you would just use a background process worker and a queue. The one time task get's added to the queue. Alternatively, you can have the heroku dyno run commands as a one off via the `heroku run ....` command. I would highly recommend looking into the `heroku scheduler` addon. That is something you can configure to run once every 10 mins/1 hour/24 hours. Might work depending on how "snappy" you need to be in running those tasks. I build a dozen or so web apps a year and Heroku scheduler works for probably 90%+ of them. – Nuclearman Oct 23 '20 at 08:03
  • As for the sleeping dyno heroku scheduler will spin up a new dyno as a one-off dyno that has access to your codebase and database. It probably won't even wake your sleeping dyno. Basically with heroku scheduler you just set it up so you can check to see if there are any tasks every 10 minutes, if so, do those tasks if not do nothing. Requires less than probably 10-30 seconds of dyno time if do nothing to be done. – Nuclearman Oct 23 '20 at 08:05
  • Thank you for your comments. I'm gonna try something like that and see how it goes. – Javain Oct 23 '20 at 08:14
  • I did manage to make it work with Heroku Scheduler. The one-off dyno checks if there's gonna be work for the next 10 minutes. If so, it wakes up the dyno by pinging. If not, it does nothing until it's executed again in 10 minutes. – Javain Nov 09 '20 at 17:51

0 Answers0