2

I have just setup a load balancer with AWS for a few instances as search engine crawlers were bringing down the site (it has millions of pages). Parts of the site allow you to login so I selected:

Enable Application Generated Cookie Stickiness

and everything works fine.

I now wonder how this will effect my SEO and the crawlers. As I selected sticky load balancing does this mean that a crawler will be stuck on one server and therefore defeat the point in the load balancer?

Any recommendations will be appreciated.

John Wheal
  • 456
  • 5
  • 17

2 Answers2

2

Since ELB uses cookie-oriented stickiness, a crawler shouldn't ever get stuck to a single instance. However, since it's obviously a crap crawler you're dealing with here (since it's DoSing your site), it might be stupid enough to hold on to cookies it gets, in which case you're going to get hosed.

Personally, I'd recommend getting rid of the session affinity and fixing your app so it doesn't need it -- there are plenty of ways to do shared session storage.

womble
  • 96,255
  • 29
  • 175
  • 230
2

Application Generated Cookie Stickiness means your app has to drop a cookie that the load balancer looks for to assign affinity. Without the cookie, it's going to be random what server the request is assigned to.

Unless you've modified your app to do drop the affinity cookie, you probably want to check "Enable Load Balancer Generated Cookie Stickiness" instead. This way the ELB itself drops a cookie for the duration value you set and keeps the user on the same server until the cookie expires.

Jason Floyd
  • 1,792
  • 1
  • 13
  • 18
  • Thanks for your answer I have solved the problem by storing the sessions on an external memcached server. – John Wheal Jul 02 '12 at 20:57
  • That's a nice, sneaky way of doing it... detect the spiders and make sure you don't give them back a sticky cookie... – womble Jul 03 '12 at 10:55