I am running VMWare ESX 4.1.0, 260247 on an intel DX58SO motherboard with a HighPoint 4310 raid controller.

The server has 2 velociraptor drives in RAID 1 and 2x WD black 2TB drives in RAID 1 as well.

The rig has been working properly for over a year, but yesterday we started noticing extremely bad disk IO on the vm's at random. Regardless of which raid the vm was stored in, we recorded 1MB/s reads.

If we restart the vm, performance goes back up to 80MB/s on reads, but in a matter of hours it goes back down to 1-3.

We've tried just leaving 1 vm on, and still experience the same issue.

I've ran into this problem before with ESXi as well, and have never found a solution. It seems to happen over time.

Thanks in advance for any feedback.

  • 1
    Do you have a lot of snapshots? Having lots of snaps can significantly degrade performance.
    – MDMarra
    May 31 '11 at 19:16
  • I don't have any snapshots..
    – 0x6A75616E
    May 31 '11 at 19:58
  • Are you running VMWare tools in the guest VM(s)? Is it a current version? What operating systems are running in the guests?
    – ewwhite
    May 31 '11 at 20:10
  • The guests are running Ubuntu server 10.04. They do have vmware tools installed. Keep in mind this set up was working properly for over a year, and suddenly started having issues.
    – 0x6A75616E
    May 31 '11 at 20:17

I've encountered a similar issue where the root cause was insufficient available RAM, forcing the host machine to make heavy use of swap space, and as a result causing the disk IO speed to drop heavily. Keep in mind that you should have around 1Gb available RAM for the hypervisor, in addition to the RAM required for the individual virtual machines.

  • Hi there, unfortunately I don't think this is the case. I have 12gb of ram in there and only about 6 are being used :(
    – 0x6A75616E
    May 31 '11 at 20:41
  • No, definitely not a RAM issue then. Out of interest, how are you performing the IO test? From within the VM's or directly on the host? May 31 '11 at 21:04
  • From the guest, im running hdparm -t /dev/sda I get 2MB/s - On the Host, I do the same and get 78MB/s :|
    – 0x6A75616E
    May 31 '11 at 21:11

Based on this VMWare Communities thread I've added this to my vmx files:

diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0" 

After starting the guests, performance seems to be back to normal.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.