If I'm buying a server, if I consider the price/performance ratio for buying more rack space, or buying more powerful servers, which is more inexpensive? Getting the most powerful box and saving a rack space, or buying less powerful servers and using more rack space.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. If this question can be reworded to fit the rules in the help center, please edit the question.
More powerful servers in smaller form factors cost more money once during their amortization cycle.
If you are renting space in someone's datacenter more rack space costs you more money on a recurring basis (monthly CoLo fees)
More powerful servers may consume more power (which can cost you more money whether you're renting space or housing it yourself), and generate more heat (which will cost you money for cooling, and if it's excessive may result in your colo facility charging you extra for the heat load).
As Jeff said, break out a spreadsheet and amortize the costs. Do what makes the most sense based on that analysis.
People are still buying their own servers? Which is more expensive will vary greatly with the data center and server cost at the time. I think it's relatively nuts to buy a non-virtualization host server, so the real question is how much will your cloud cost, and where is the cheapest place to store the CPUs you need.
EDIT: Cloud may or may not be expensive- (thus the question how much will it cost - and generally speaking virtual is cheaper than physical per machine) Load has little to do with whether of not to use virtualization (as @synetcon-dj mentioned there isn't much overhead for CPU). I've done quite a few virtualization migrations for financial analysis servers, SAP servers etc., so I'm used to the types of load you are talking about. In your case had you been running a virtual environment you could have utilized the resources of your other servers. The entire point of virtual environments is to pool resources, so before I'd sign off on a budget adding more cores, you'd better be able to show that all of the cpus we already have are at 80%. It simply does not make sense to have unutilized or underutilized servers in an IT environment unless you are an extremely small shop (in which case I'd ask why you aren't looking at a service based solution).