I offer my solution to the following problem, and ask you networking and server admin professionals to validate it or poke holes in it. I am interested in any obvious attack vectors or scalability issues you may see. Thanks!
Requirements:
- HTTPS support, handled by each application server independently
- near-linear horizontal scalability
- bandwidth distributed across servers (response data does not all return through the LBs or proxies)
- something like failover for application servers and load balancers
- client-server affinity
- Linux-friendly (solution not closed source)
- bootstrap-friendly! (i.e. low initial cost)
The Scheme:
PUBLIC NETWORK
+-----+------+--------+-----+------->
| | | |
v v v v
+---+ +---+ +--+ +--+
|LB1| |LB2| ... |S1| |S2| ...
+---+ +---+ +--+ +--+
Redundant load balancers (LB*, via something like DNS RR, or just failover): their only purpose is to offer clients the URI to some application server instance, which the client would then use perpetually for its requests. The distribution would be random or round robin, initially.
Application Server instances (S*) each independently handle requests directly from clients.
Stateless architecture lets individual servers go down. Clients request a new server from the load balancers if their assigned server fails.
New application servers could spin up, register with the load balancers, and be assigned to clients very quickly. All S* would have a subdomain DNS entry to share a wildcard certificate.
A naive implementation could be done entirely on one server with zero redundancy, and delegate responsibilities to expand as needed.
Immediate Concerns
Firewall and DDoS protection would have to be managed at each server, instead of centrally like you have with load balancing reverse proxies. Centralized configuration management is as far as I've thought into this.
This scheme does not take advantage of geographic location or server response time, as something like Anycast DNS would. It's a conscious trade-off for greater likelihood of server affinity, and can possibly be shoehorned in later.