Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am running VMWare ESX 4.1.0, 260247 on an intel DX58SO motherboard with a HighPoint 4310 raid controller.

The server has 2 velociraptor drives in RAID 1 and 2x WD black 2TB drives in RAID 1 as well.

The rig has been working properly for over a year, but yesterday we started noticing extremely bad disk IO on the vm's at random. Regardless of which raid the vm was stored in, we recorded 1MB/s reads.

If we restart the vm, performance goes back up to 80MB/s on reads, but in a matter of hours it goes back down to 1-3.

We've tried just leaving 1 vm on, and still experience the same issue.

I've ran into this problem before with ESXi as well, and have never found a solution. It seems to happen over time.

Thanks in advance for any feedback.

share|improve this question
1  
Do you have a lot of snapshots? Having lots of snaps can significantly degrade performance. –  MDMarra May 31 '11 at 19:16
    
I don't have any snapshots.. –  thatjuan May 31 '11 at 19:58
    
Are you running VMWare tools in the guest VM(s)? Is it a current version? What operating systems are running in the guests? –  ewwhite May 31 '11 at 20:10
    
The guests are running Ubuntu server 10.04. They do have vmware tools installed. Keep in mind this set up was working properly for over a year, and suddenly started having issues. –  thatjuan May 31 '11 at 20:17
add comment

2 Answers

up vote 0 down vote accepted

Based on this VMWare Communities thread I've added this to my vmx files:

diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.dataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0" 

After starting the guests, performance seems to be back to normal.

share|improve this answer
add comment

I've encountered a similar issue where the root cause was insufficient available RAM, forcing the host machine to make heavy use of swap space, and as a result causing the disk IO speed to drop heavily. Keep in mind that you should have around 1Gb available RAM for the hypervisor, in addition to the RAM required for the individual virtual machines.

share|improve this answer
    
Hi there, unfortunately I don't think this is the case. I have 12gb of ram in there and only about 6 are being used :( –  thatjuan May 31 '11 at 20:41
    
No, definitely not a RAM issue then. Out of interest, how are you performing the IO test? From within the VM's or directly on the host? –  Richard Keller May 31 '11 at 21:04
    
From the guest, im running hdparm -t /dev/sda I get 2MB/s - On the Host, I do the same and get 78MB/s :| –  thatjuan May 31 '11 at 21:11
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.