The load balancer isn't doing anywhere near as much processing as each one of those backend servers.
It's possible to DDoS a load balancer, the same way it's possible to DDoS any other server, but the load balancer itself is really just checking the "Host: " header, and / or handling the SSL sessions (assuming you're also using the load balancer as the SSL endpoint), whereas the actual app servers need to handle the app logic, read from the drive, the database, etc.
If the app servers were just serving the same string for every connection, then yes, the load balancer would be kind of pointless.
Interestingly, some load balancers don't even need to check application-level requests. Thanks to Direct Server Return (DSR), you can load-balance at the IP level. This has advantages and disadvantages; on the one hand, application acceleration techniques (such as SSL session handling) cannot be done at an IP-level load balancer. On the other hand, the load balancer doesn't need to serve response packets (except for, perhaps, ICMP), so it doesn't really need much outgoing throughput, and (as before) it only needs enough incoming throughput to handle incoming application traffic.
Then again, if you're really dealing with an enormous amount of traffic, you can employ all these methods simultaneously; you can use DNS to distribute load across clusters, each cluster can have an IP-level reverse proxy backed by application-level reverse proxies.