Let's say I'm using Amazon's EC2 load balancer. I have it hooked up to two instances (excuse me if my terminology isn't correct). What happens if the load balancer fails? Do both instances fail to work now?
Typically load balancers are clustered together into a high-availability pair.
If one load balancer fails, the secondary picks up the failure and becomes active. They have a heartbeat link between them that monitors status.
If all load balancers fail (or are accidentally misconfigured), servers down-stream are knocked offline until the problem is resolved, or you manually route around them.
Yes if the load balancer instances both fail the service is essentially dead. Note that amazons uptime includes up to 4 hours of downtime a year (99.95%)