Scheduling in Heroku while keeping track of counter

Question

Might be a noob question but I'm new to hosting code online. For my first project I've created a web scraper that logs into a website and updates a number every hour. Indexing up by one. Initially I was scheduling using the schedule library. Looked a little like this:

import schedule 

int number = 4

def job():
      number = number + 1
      element.clear()
      element.send_keys(number)


schedule.every().hour.at(":00").do(job)

while True:
     schedule.run_pending()
     time.sleep(1)

This worked fine for the first 24 hours until the dyno had to be cycled. After which, the value number reverted back to 4. I could use some built-in scheduler but I don't see how that would remedy the issue since it will run the int number = 4 line anyway. What am I missing and how can I keep track of the running count even when the dyno resets?

Answer 1

This isn't really about schedulers, it's about where that data lives. Right now it is in memory and (a) will be lost whenever your dyno restarts and (b) will not behave properly if you scale your application up or out.

You need to store it somewhere.

Heroku offers a number of addons that do data storage, many of which have free tiers.

Depending on your use case, you could also store it on an off-site blob storage container like Amazon S3 or Azure Blob Storage, but a data store is likely a better choice.