Dan Carley's answer is good -- I'm expanding on a misunderstanding I often see people make.
2-node active-active app server cluster for a HA scenario. ... when one of the server goes down, the entire site load will come on one server
Yes, which is why 2-node active/active often is a poor solution. At a high level, if you have 2 nodes you can do one of the following:
- 2 node active-active, where each single node cannot handle full daily load. By employing 2 nodes, you get a certain extra headroom for load spikes (slashdotting) and you can take a single node offline during the night for maintenance without service disruption. You do not get high availability, since the failure of a single node during peak hours will take your system down. In fact, you have at least 2 single points of failure for your service (each node).
- 2 node active-passive, where each single node can handle full load. You get high availability, and relatively simple setup.
- 2 node active-active, where each node can handle full load. You get all of the benefits of 1) plus high availability, but typically at the cost of a more complex setup and more expensive hardware requirements.
For a 2 node HA system active-passive is generally my preferred setup. There is IMHO little benefit in running active-active in a 2-node setup, and active-passive is often easier to understand and simpler to get working. Active-active becomes a good solution when you have 3 or more nodes, so that fail over of one nodes traffic to the others is not a problem.
I'm not 100% clear on what OP means, but I think his option 2 corresponds to my option 2 or 3, in the sense that each single node runs at less than 50% capacity.