I am running my app using an AWS setup - Tomcats running behind an Elastic Load Balancer (ELB) on EC2 instances, a memcached-backed Elasticache with a couple of nodes. I've configured my web application to store session data in Elasticache in order to keep the Tomcat servers highly available to the user (i.e. when a Tomcat shuts down, the user isn't logged out, but their requests merely get served by another available Tomcat). It works as expected, except for one interesting case I noticed during testing.
When I shut down the Tomcat that is currently running the app, after a moment, another Tomcat starts serving the requests and the user stays logged in. However, when I restart the stopped Tomcat, the app switches back over from its current Tomcat to run on the previously stopped instance, which is not what I'd expect - I thought that the app would continue running on its new Tomcat until it was stopped, and then it would try to switch again.
I've looked around for an explanation for this behaviour, and some sources have indicated it could be an ELB configuration problem, but fail to mention what configuration option could be causing this "preferential primary" treatment. My ELB is currently configured for sticky sessions, and uses the AppCookieStickinessPolicy with a cookieName of JSESSIONID. So far, all my Tomcats reside in the same availability zone (us-east-1b). Any ideas? Is this stickiness behaviour typical?
EDIT: Amazon's documentation here: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-sticky-sessions.html seems to directly refute the behaviour I've observed.
If an instance fails or becomes unhealthy, the load balancer stops routing requests to that instance, and chooses a new healthy instance based on the existing load balancing algorithm. The load balancer treats the session as now "stuck" to the new healthy instance, and continues routing requests to that instance even if the failed instance comes back.
