You mostly have it right, but one detail you kind of glossed over is the response part. The setup you describe is called "Direct Routing" meaning that the packets come in to the load-balancer, get forwarded on to a back-end server, and that server replies directly to the client without passing the traffic back through the load-balancer.
In order for DR to work, the load-balancer IP address needs to ALSO exist on all the web servers. However, you need to disable ARP responses for those addresses on the other servers. Here is a reference to a discussion of this on CentOS.
The other thing to remember is that this proxy can now become a bottle-neck or a single point of failure. Usually when these load balancing proxies are set up, they are set up as a High Availability cluster of machines so that maintenance can be done on one or one can fail without taking the whole site down.
Beyond DR, you can also do NAT, where the web servers use the load-balancer as the gateway, and the web servers have a private IP, which the load balancer translates to and from the public IPs. This is generally easier to configure, because you don't need to worry about the ARP issues or asymmetric routing, etc...
Finally, one method not frequently spoken about is using the iptables CLUSTERIP module. This module blocks traffic based on the remote IP address or IP address and port number, and an idea of what node in the cluster it is running on. So, you'd configure the IP address on all the machines as an alias, and configure CLUSTERIP. It uses a hash of the connection information so that every machine in the cluster agrees which node will handle it. Packets are blocked on the nodes that don't handle it, and are accepted by the node that does. This is done using a multicast MAC address at the low level.
This works great because you don't have a dedicated load-balancer to fail. However, it is somewhat primitive and obviously cannot do load-balancing based on URLs or other "layer 7" information, only based on IP address.
Here's an article I wrote on how to set up CLUSTERIP last year.
There are many choices and we use them all, depending on the situation. They all have their strengths and weaknesses, depending on exactly your situation and goals and level of experience.