I have installed debian 7 on a physical machine. This is the configuration of the machine:
- 3 hard drives using RAID 5
- Strip element size: 1M
- Read policy: Adaptive read ahead
- Write policy: Write Through
- /boot 200 MB ext2
- / 15 GB ext3
- SWAP 10GB
- LVM rest (~500GB)
I installed postgresql, created a big database (over 1GB). I have an SQL request that takes a lot of time to run (a SELECT statement, so it only reads data from the database). This request takes approximately 5.5 seconds to run. Then, I installed XEN, created a domU, with another debian distro. On this OS, I also installed postgresql, with the same database. The same SQL request takes only 2.5 seconds to run.
I checked the kernel on both dom0 and domU. uname-a returns "Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2 x86_64 GNU/Linux" on both systems. I checked the kernel parameters, which are approximately the same. For those that are relevant, I changed their values to make them match on both systems using sysctl. I saw no changes (the requests still take the same amount of time).
After this, I checked the file systems. I used ext3 on domU. Still no changes.
I installed hdparm, and ran hdparm -Tt on both systems, on all my partitions on both systems, and I get similar results.
Now, I am stuck, I don't know what is different, and what could be the cause of such a big difference.
- Debian runs on a Dell server PowerEdge 2950
- postgresql: 9.1.9 (both dom0 and domU)
- xen-linux-system: 3.2.0
- xen-hypervisor: 4.1
As Krzysztof Księżyk suggested, it might be due to some file caching system. I ran the dd command to test both the read and write speed.
Here is domU:
root@test1:~# dd if=/dev/zero of=/root/dd count=5MB bs=1MB ^C2020+0 records in 2020+0 records out 2020000000 bytes (2.0 GB) copied, 18.8289 s, 107 MB/s root@test1:~# dd if=/root/dd of=/dev/null count=5MB bs=1MB 2020+0 records in 2020+0 records out 2020000000 bytes (2.0 GB) copied, 15.0549 s, 134 MB/s
And here is dom0:
root@debian:~# dd if=/dev/zero of=/root/dd count=5MB bs=1MB ^C1693+0 records in 1693+0 records out 1693000000 bytes (1.7 GB) copied, 8.87281 s, 191 MB/s root@debian:~# dd if=/root/dd of=/dev/null count=5MB bs=1MB 1693+0 records in 1693+0 records out 1693000000 bytes (1.7 GB) copied, 0.501509 s, 3.4 GB/s
What can be the cause of this caching system? And how can we "fix" it? Can we apply it to dom0?
I switched my virtual disk type. To do so I followed this article.
I did a
dd if=/dev/vg0/test1-disk of=/mnt/test1-disk.img bs=16M
Then in /etc/xen/test1.cfg, I changed the disk parameter to use file: instead of phy:
it should have removed the file caching, but I still get the same numbers (domU being much faster for Postgres)