Imagine we have a site made of nginx as a frontend, and apache (with php or whatever) as a backend. The idea is that nginx will serve static while proxy requests to *.php files to backend. Easy to implement and no-brain-needed config, isn't it?
Now imagine we have bad-behaviour backend. It uses some unstable modules, it used to produce pages for seconds and we're not sure if the page be rendered at all. Weird, but still possible, right? Legacy code and so on. By the way, the backend may be on different machine (yes, legacy: old OS, old software, "don't even touch it!" etc.)
Now we'd like to "protect" ourself in a case the backend won't answer. It would be great to serve page as we served it last successful time when we got no page or get 5xx error. To say in other words, if backend returned 200 code and the page content, then we serve it. If we got timeout or we got 5xx code from backend, we serve the page for the same URI but cached one. This will give us some time to fix backend while users will see at least something.
Nginx is a nice piece of software. I found proxy_cache_use_stale directive and also some others but I'm afraid what will be the version that will be served if backend fails.
I can also imagine some http balancer (nginx as an example) that will check the backend status and stop use it whatever happens. The second backend then maybe server full of static pages that mimics the original site (maybe even without some element, like 'login' link). If so, how should I wisely update the 'mimics' server over time?
What approach may be better? What would you choose?