Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

So, when using AWS ELB, there's a health check url we use to ping db server / check out bound connection, etc.

If there are unhealthy instances, they are removed. But if the db server is down, our web server is designed to continue function to the best of its ability. (Design for failure).

These 2 concepts seems to conflict each other? If a single server is not health, it'd be removed. If all server is affected, we want to keep them online.

How to solve this dilemma?

share|improve this question
    
Why are you pinging your db server for a health check? Instrument your health check so that it checks your app servers and so that it fails if your app servers aren't able to get a response from your database. – EEAA Jun 24 '14 at 2:43
    
@EEAA that's what I'm doing and what i mean – Sleeper Smith Jun 24 '14 at 3:24
    
You said that you're pinging your db server for your health check. You should not be doing that. – EEAA Jun 24 '14 at 3:35
    
@EEAA i meant from the web server. The webserver test it's connection to the dB server. If the web server can't see the dB server, it goes off line – Sleeper Smith Jun 24 '14 at 7:03
    
ELB Healthcheck is designed to provide a degree of availability in the EC2 (compute) capacity behind it. If your compute cluster has no means to provide service without accessing the data layer then you must ensure that the data layer provides an availability level at least as good as the compute layer and focus your monitoring efforts on it. – ma.tome Apr 29 '15 at 8:16

Those two concepts, as describe, do conflict with one another. The ELB needs to get HTTP 200 responses to the health check or the instances will be dropped from the ELB.

If you want the application to stay online even after the DB has failed you'll want to change the health check URL to something that will still respond with HTTP 200s even when the database is down.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.