2

I support an ERP system for a manufacturer. We're experiencing intermittent application performance issues on our database, hosted on VMware by a third-party cloud provider. (They're a smaller provider... not Google, Microsoft, Amazon, or the like.) It's always on the back of my mind that their business interests dictate shoving as many customer VMs onto a single host as they can get away with, and I don't know if we're losing performance due to long CPU ready times, memory ballooning, or disk resource contention. Their salespeople paint a rosy picture, of course, but if I'm chasing down other unknowns, how can I be certain where the problem lies? How can I rule out the infrastructure layer of the stack without access to the host machine? I'm seeing periods of 100% CPU usage on every core, and no obvious explanation can be found when reviewing the running database sessions. (Sometimes jobs run quickly, and sometimes nearly-identical jobs hang with no apparent blocks.)

Migrating to another provider or back-sourcing these machines to our own server room are both unlikely in the immediate future, but it would be nice to have hard evidence to drive our next action from here.

Edit: The guest machine is running Windows 2008 R2 Datacenter.

2
  • 2
    You should monitor CPU steal time; this will give you some indication of whether the host CPU is significantly overprovisioned. May 20 '16 at 18:05
  • Ask if they're using vCloud Director, if it's a recent version they can provide you with all manner of stats
    – Chopper3
    May 20 '16 at 18:35
4

It's perfectly possible; pretty much anything can be specified in a Service Level Agreement (SLA). Since your provider is a smaller one, I would say you have a somewhat better chance of getting this into your contract than with a larger provider.

However, some caveats:

  1. A meaningful SLA specifies a) what will be provided, b) how it will be measured, and c) what will happen in case of breach. Make sure that you're happy with all three; an SLA without any of these is not worth the paper it's written on. The cost of a breach (c) should be large enough that it provides a real commercial incentive for your provider to avoid breach.

  2. You will not be able to insert this into your contract mid-term (unless your provider is amazingly stupid^Wobliging). Renewal is the time to address your requirements going forward.

  3. Your request will likely be ignored unless you're prepared to leave your current provider if they won't sign the new contract you want, and leaving will be commercial suicide unless you have somewhere to move to. So now is the time to start doing due-diligence on a new provider who will give you what you want, so that you have some teeth in any negotiation.

  4. This is going to cost real money. Most hosting contracts lack at least one element of a meaningful SLA (see above), and therefore promise only a best-effort attempt to provide some kind of service. Actually honouring the terms of a meaningful SLA costs your provider real money; you must expect the costs they incur in meeting your particular requirements to be passed onto you - they're not in business for the good of their health. Doing your due-diligence now will give you some idea of what this is likely to cost you, so you can prepare management for the real cost of the uplifted service.

1

You can not. Especially if you have not contractually agreed to things like dedicated CPU cores you actually DO get what you pay for. There is no ground of a complaint as you et what you ordered.

Review your contract. What is a CPU core according to the contract?

2
  • We'll review that, but meanwhile, provided the contract says our cores are dedicated (1:1 virtual to physical cores), is there a technical solution to validate whether they're holding their end of the bargain?
    – durette
    May 20 '16 at 18:06
  • No, not really. Sad - sort of - but the whole idea is to measure isolation of the VM. You COULD run something like a performance test over a longer duration (and check how your CPU cores "sum up" against the real thing) but that would render the VM useless during measurements.
    – TomTom
    May 20 '16 at 18:09
1

If you have VMware Tools installed then there is a number of performance counters that you can check (in perfmon.exe see the categories "VM Processor" and "VM Memory") to find out if your virtual CPU cores really always get the 100% equivalent of a physical core. This way you can also check for CPU and Memory reservations and limits, ballooning etc.

Parts of this information is also exposed through a CLI tool that is included in VMware Tools, see

"c:\Program Files\VMware\VMware Tools\VMwareToolboxCmd.exe" help stat

If your contract states that your virtual CPU cores are "dedicated" then a CPU reservation should be in place that you can check in the above mentioned ways.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.