2

I have just setup a load balancer with AWS for a few instances as search engine crawlers were bringing down the site (it has millions of pages). Parts of the site allow you to login so I selected:

Enable Application Generated Cookie Stickiness

and everything works fine.

I now wonder how this will effect my SEO and the crawlers. As I selected sticky load balancing does this mean that a crawler will be stuck on one server and therefore defeat the point in the load balancer?

Any recommendations will be appreciated.

2 Answers 2

2

Since ELB uses cookie-oriented stickiness, a crawler shouldn't ever get stuck to a single instance. However, since it's obviously a crap crawler you're dealing with here (since it's DoSing your site), it might be stupid enough to hold on to cookies it gets, in which case you're going to get hosed.

Personally, I'd recommend getting rid of the session affinity and fixing your app so it doesn't need it -- there are plenty of ways to do shared session storage.

1
  • Thanks for your answer I have solved the problem by storing the sessions on an external memcached server.
    – John Wheal
    Jul 2, 2012 at 20:57
2

Application Generated Cookie Stickiness means your app has to drop a cookie that the load balancer looks for to assign affinity. Without the cookie, it's going to be random what server the request is assigned to.

Unless you've modified your app to do drop the affinity cookie, you probably want to check "Enable Load Balancer Generated Cookie Stickiness" instead. This way the ELB itself drops a cookie for the duration value you set and keeps the user on the same server until the cookie expires.

2
  • Thanks for your answer I have solved the problem by storing the sessions on an external memcached server.
    – John Wheal
    Jul 2, 2012 at 20:57
  • That's a nice, sneaky way of doing it... detect the spiders and make sure you don't give them back a sticky cookie...
    – womble
    Jul 3, 2012 at 10:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .