We're developing our web application and have been working on reducing our load times. When we began developing the application, we signed up with a well known hosting provider offering "dedicated solutions" on the cloud. As a result, we have one dedicated server for our web application and one dedicated server for our database. Both servers were setup with the same amount of RAM, the same CPU, and SSD drives. Even with investing time & money into our infrastructure, we noticed 8-10 second load times.
We were advised by another hosting company that we should scale down and put everything onto one server (instead of splitting it up). They mentioned that having the servers separate would lead to higher load times because of network latency and stated that PHP would communicate quicker with SQL via sockets than over the network. This I know to be true, but I did not expect the results we got. Immediately after moving everything to our original application server, our load times dipped to 3-4 seconds from 8-10!
The problem is that we're now looking into a new hosting provider and have been advised to utilize a cluster with a load balancer, database server, application server, and scale from there. The concern is that if we separate the application and database server again, that we'll be back to square one.
From everything I've read, it seems that it's almost always recommended to have these servers split rather than grouped together. Is there a performance boost that comes with doing this when it is setup properly, or is it merely for scalability in the long run?
I appreciate the assistance!