I'm not sure why you would want a time-out or say 50X response to be a valid one, but if you really just want 404's, then the Nginx docs explain it this way:
http://nginx.org/en/docs/http/ngx_http_upstream_module.html#server
max_fails=number
sets the number of unsuccessful attempts to communicate with the
server that should happen in the duration set by the fail_timeout
parameter to consider the server unavailable for a duration also set
by the fail_timeout parameter. By default, the number of unsuccessful
attempts is set to 1. The zero value disables the accounting of
attempts.
Here's the kicker:
What is considered an unsuccessful attempt is defined by the
proxy_next_upstream, fastcgi_next_upstream, uwsgi_next_upstream,
scgi_next_upstream, and memcached_next_upstream directives.
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream
Syntax: proxy_next_upstream error | timeout | invalid_header | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | off ...;
http_404
a server returned a response with the code 404;
This answer to a similar post could form the base for your config:
https://stackoverflow.com/a/12878310/4995465