I am running PHP in preforked(MOD_PHP) on an apache2 server. The setup is on linux Ubuntu 10.04. The database I am using is a firebird database 2.5.2. The apache2 server are running in a web cluster consisting of 8 webservers.
At a point in time we had serious performance issues due to large spikes in the requests to the application(we have peek hours). The bottleneck showed to be the amount of database connections needed to resolve the number of requests that came in a very short time. Firebird didn't cope with it very well and requests simply timed out.
This type of database does not have a connection pool and that is why I have been using pconnect in PHP to be able to relieve some stress from the database. This persists the database connection in the apache2 process. This was a major performance boost. The down side is that we had to let an apache2 process take a lot of requests before it gets rotated and we keep a lot of apache2 process running even though there is not load for it. The webservers are running with 70 apache processes each. This is to keep the connections open and ready. Basically we tried to make apache2 our connection pool. This works. When a user requests the application the database handle is ready and Firebird doesn't have to worry about the cost of creating a new database connection.
Now here is my question. We now need to have a lot of databases - small ones. But they will all be running within the cluster of apache2 servers. This means that in the lifetime of an apache2 process its very likely to get persistent connections to several databases (maybe 80-100).
I'm concerned how apache2 will handle this scenario. Is there a limit in apache2 in how many connections it can handle? Will it get slower.. will it just grow in memory and handle everything perfect?
For now there is nothing to do about the database sharding. We (as the dev team) were not at all fond about the idea of splitting the databases. But there were no green light for rewriting the application and create a new database structure to get more performance from the code. Hardware, for now, is the answer. There are also legal matters that forces us to split into several databases to enclose data. But this is were I get a bit worried about what apache2 is able to handle.
Does anyone know?