1

I have a python program that I would like to constantly be running updates and gathering new data. Essentially, I am gathering data from a bunch of domains. My processors take about a day and a half to run. Once they finish, I'd like them to automatically start over again.

I don't want to use a while loop to just restart the processes without killing everything related first because some of the packages that I am using to support these processors (mainly pyV8) have a problem of memory slowly accumulating and I'm not a good enough programmer to dive into debugging a memory leak in a big package like that. So, I need all of the related processes to successfully die and then come back to life.

I have heard that supervisord can do this type of work, but don't like messing around with .conf files and would prefer to keep everything inside of python.

Summary: Is there a package that will kill all related processes with a script/package that I could use to put into a while loop or create this kind of behavior inside of a python script?

DilithiumMatrix
  • 17,795
  • 22
  • 77
  • 119
ccdpowell
  • 629
  • 5
  • 14
  • 22

1 Answers1

2

I don't see why you couldn't use supervisord. The configuration is really simple and very flexible and it's not limited to python programs.

For example, you can create file /etc/supervisor/conf.d/myprog.conf:

[program:myprog]
command=/opt/myprog/bin/myprog --opt1 --opt2
directory=/opt/myprog
user=myuser

Then reload supervisor's config:

$ sudo supervisorctl reload

and it's on. Isn't it simple enough?

More about supervisord configuration: http://supervisord.org/subprocess.html

Jan Spurny
  • 5,219
  • 1
  • 33
  • 47
  • This is a pretty simple solution and I asked the question more out of curiosity on how to execute and completely close python files from within python. Thanks for the summary on supervisord. it is pretty straight forward and will likely be what I end up using. – ccdpowell Apr 30 '15 at 19:30
  • Ended up just going with supervisord. Really easy and powerful. – ccdpowell Apr 30 '15 at 23:10