I want to use apache load balancer so that one tomcat server is primary, and the secondary server is online only if the first server is down. how can this be done?
If you're only going to do load balancing, I would recommend something more lightweight than Apache, such as HAproxy, but lets assume that you're going to go with apache.
There are several methods for load balancing, so you should spend a little bit of time getting acquainted with them. There is an excellent article available by Willy Tarreau called Making applications scalable with Load Balancing that is worth your time to read.
There are a couple of tools for making Apache a load balancer, and maybe the easiest (that I know of) is by using mod_proxy_balancer. The only problem with this is that it doesn't appear to do failover load balancing (which is what you want: high availability, rather than true load balancing).
Since you're using Tomcat, why not use mod_jk, the Apache tomcat connectors?
Assume for a second that we're on a Redhat-based system (I'm using CentOS 5.5) that is configured to compile things (for instance, has gcc and the appropriate libraries (this isn't a good idea on production systems. This is not the time or place to discuss RPM packaging, though)
This will install Apache along with the development stuff for making modules, and the configuration directory will be /etc/httpd/
The current (20110130) Tomcat Connectors are 1.2.31 (but you can check for the latest here), so download them:
Extract, compile, and install them:
Verify that mod_jk.so was installed:
Now, we get to actually configure it.
Alright, that tells Apache that we want to use mod_jk, that we want to use a file called /etc/httpd/conf.d/workers.properties, and that we have one virtualhost where everything is sent to the workers. Great, now lets configure the workers. Edit /etc/httpd/conf.d/workers.properties:
Now, we have two actual workers, and one virtual load balancer for them. You can now start Apache:
Test it out by browsing to the machine, and it should go to the first tomcat server. You should be able to verify this by watching /var/log/httpd/mod_jk.log (or wherever you pointed it).
Here is a much simpler solution using AJP instead of JK:
There are quite a few more configuration properties you can set, such as differential load factors if you have more backends, different balancing algorithms, balancer sets, etc: see the Apache mod_proxy documentation, but the above does exactly what you specified. -D means 'not disabled' and +H means 'hot standby'. And you can swap them at runtime too.
You just need to run two Tomcats, with AJP connectors running on port 8009 with address="127.0.0.1", address="127.0.0.2" etc, as many as you need, with