21

I have googled this question for a while but can't find the answer. My question is while deploying the nodejs application on aws elastic-beanstalk servers, I want the nodejs application would be restart automatically if the application crash.

Actually there're few nodejs package already support this by command line, such as forever, but there's no easy way from console management or awscli to install this package and execute it to achieve restart automatically.

I am wondering how do you resolve the restart issue on aws eb?

HolaMan
  • 381
  • 1
  • 4
  • 11
  • Sorry, I deleted my answer because it does not look relevant to the question after additional consideration. I can suggest to create a tiny `forever`-powered wrapper for your application and then deploy it. – Alex Netkachov Nov 07 '13 at 17:21
  • 1
    I'm not 100%, but I'm pretty sure EB restarts the application for you. Anyone care to confirm or deny this? – Eric Feb 07 '14 at 19:31
  • 1
    @Eric I too want to know this. – winduptoy Jan 07 '15 at 05:05

6 Answers6

34

I've confirmed (as of Mar 11, 2015) that EB does indeed restart node for you.

To confirm, I added a hidden query param to my app:

if (req.query.testcrash == 'true') {
    setTimeout(function(){
        undefined.crashMe();
    }, 100);
}

Then verified in the log that the exception happened, and that my app was restarted.

For reference:

  • My EB/EC2 config is "64bit Amazon Linux 2014.09 v1.0.9 running Node.js"
  • Using nginx and node 0.10.31
ryanman
  • 3,794
  • 3
  • 25
  • 15
  • 1
    Yeah, I can confirm that the node server auto-restarts on crash – jiawen Apr 21 '16 at 02:47
  • 2
    Any additional link or document which elaborates this behavior and how it's internally managed? – Akash Budhia Oct 17 '16 at 12:42
  • 3
    It does this by simply using the upstart `respawn` option. By default it's set with a respawn limit of maximum 30 restarts with 60 seconds, if the process restarts more times than that within that time, it will not be started again until done manually. – Cybolic Jun 18 '17 at 02:29
  • 1
    @Cybolic: Could you provide a reference, or explain how you learned of this respawning up to 30 times in 60 seconds? – Rob Johansen Mar 12 '18 at 21:45
  • 3
    @RobJohansen: It's in the upstart script at `/etc/init/nodejs.conf`. There are two lines, `respawn` and `respawn limit 30 60`. The documentation for this in upstart is at , which states: `respawn limit COUNT INTERVAL | unlimited Respawning is subject to a limit. If the job is respawned more than COUNT times in INTERVAL seconds, it will be considered to be having deeper problems and will be stopped.` – Cybolic Mar 14 '18 at 10:07
9

Add forever to your package.json so it gets installed automatically. Then in EB console, under configuration, custom node command:

node_modules/.bin/forever app.js
davychiu
  • 91
  • 2
3

Yes, better option to use Supervisor, however in order to have ability to restart app server with help of aws console or beanstalk cli tools you need to put own handler to Elastic beanstalk hooks in the directory: /opt/elasticbeanstalk/hooks/restartappserver/enact Hook is shell, python or ruby script that placed in mentioned directory. Put logic of the supervisord restart here and you will be able to restart it with help of management console, aws cli tools (http://docs.aws.amazon.com/cli/latest/reference/elasticbeanstalk/restart-app-server.html), elastic beanstalk api: (http://docs.aws.amazon.com/elasticbeanstalk/latest/APIReference/API_RestartAppServer.html)

How to add hook, install supervisiord etc you can read here: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html

Vadym Fedorov
  • 2,395
  • 2
  • 21
  • 31
2

After playing around with this a bit, and inspecting the process immediately after running 

aws elasticbeanstalk restart-app-server --environment-name my-env

from @Human Love 's comment. I found these two commands for manually starting/stopping the process when ssh'd into the EC2. Not sure if these are recommended, but for quick debugging I find them useful 

# to start the process
python /opt/elasticbeanstalk/containerfiles/ebnode.py --action start-all
# to stop the process
sudo python /opt/elasticbeanstalk/containerfiles/ebnode.py --action stop-all

[NOTE]: this is a nodejs specific solution. Though other application types are probably pretty similar. To inspect the exact command. Open two terminal windows and

  1. in the first, run aws elasticbeanstalk restart-app-server --environment-name my-env
  2. in the second, run ps aux | grep python (I grepped for node because it was a node app)

to find the specific /opt/elasticbeanstalk script

Jeremy
  • 1,717
  • 5
  • 30
  • 49
1

If you want to restart the server from cron then you could use these commands.

aws elasticbeanstalk restart-app-server --environment-name my-env

Reference

Ali Nouman
  • 3,304
  • 9
  • 32
  • 53
  • This isn't the most ideal answer for the question. The OP wanted to automatically restart after a crash. A cron would be a timed event. That being said, you helped me out a bunch. I was looking for a way to restart the application on demand, while I'm doing testing. Thanks! – BoomShadow Mar 08 '17 at 21:21
  • @BoomShadow You are welcome! I'm glad that you have found the answer in some way useful. Many thanks! – Ali Nouman Mar 09 '17 at 09:52
1

Yes it does. I know because after ssh'ing in to the box i did:

sudo pkill node

And then I can verify that:

  1. the app continues to work from the browser
  2. It prints restarting to logs visible via: sudo tail /var/log/web.stdout.log -f
  3. A new node process with new process id takes its place visible via: pgrep node
user566245
  • 4,011
  • 1
  • 30
  • 36