My main concern is since reverse proxy routing is static, how can it replace load balancer which routes the request dynamically upon checking the heath of the server? Then do we really need a reverse proxy?
Every load balancer that operates at layer seven (http) is a reverse proxy, but not every reverse proxy is a load balancer. You could say that a load balancer is a type of reverse proxy.
Load balancers that work at layer four (eg AWS NLB) or below are probably also reverse proxies, but since they don't parse requests like http packets they're not as functional and have fewer features. They're usually faster.
A load balancer's primary job is to take requests and distribute them to a number of servers to service the request. It may also do things like path based routing, so for example static resource requests are filled from one server farm or AWS S3, while application pages are filled by another server farm.
A reverse proxy, if it's not a load balancer, can be installed on a single server to send requests to another application on the server. For example, you may have Nginx or Apache in front of Tomcat, as they have more features than Tomcat and can protect Tomcat from some classes of attacks. For example, Apache may be configured to cache Tomcat responses if for some reason you don't want to do that in Tomcat.