2

I am working on a project that launches a process via a Rails worker that is very resource intensive and it can only be handled properly by a Performance Worker on Heroku, 1X workers are killed because they use too much RAM and 2X workers can barely handle the load exceeding their RAM limits by up to 160%. A performance worker does the job fine with no issues.

My question is, is there a way to dynamically switch the Dyno size to Performance before a job initiates and then scale it back down once the job is finished or a queue is empty?

I know HireFire exists but to my knowledge this service only increases the amount of workers based on a queue length etc? Another possible solution I thought about was using the Heroku API which has a Dyno endpoint to resize the worker dyno before the job starts and then resize it back down when the job ends.

Does anyone else have other recommendations, ideas or strategies for this issue?

Thanks!

rii
  • 1,578
  • 1
  • 17
  • 22

2 Answers2

1

The best way is the one you mentioned: use the Heroku Platform API to scale your Dyno size up before starting the job, and then down again afterwards.

This is because tools like HireFire only work by inspecting stuff like application response time, router queue, etc. -- so there's no way for them to know you're about to run some job and then scale up just for that.

rdegges
  • 32,786
  • 20
  • 85
  • 109
  • Thanks, yeah I thought this would be one of the only solutions. Heroku suggested "Managing this via the platform API is one way to do it but if you'd like an alternative you might consider using the heroku toolbelt buildpack which will give you access to the toolbelt inside a dyno. Sorry that I don't have an easier solution for you straightaway." – rii Aug 02 '16 at 17:49
1

Depending on the specifics of the usage, you may be able to just create a distinct dyno-type in your procfile that only runs this particular worker and is always scaled to performance, but isn't always running? You could even just run this with one-off runs, instead of scaling it potentially (this can also be done via the API, roughly equivalent to heroku run ...). That said, @rdegges answer should certainly work.

geemus
  • 2,522
  • 16
  • 22