Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The problem I am facing is on my Debian box (HP MicroServer N40L) running ZFS, which is compiled from source downloaded from, seems to perform poorly.

The pool is created from 3 identical 2TB harddrive (Seagate, SATA, 5900rpm, 6GB/s). System is running on another 250GB (7200rpm, unraid) hard drive.The command I used to create my zpool is zpool create tank raidz hd1ID hd2ID hd3ID. zpool status tank indicates all is well. After I got suspicious, I used bonnie (version 1.96) to test the performance bonnie -u root:root -x 5 -s 4096 and here is some of the numbers I got back (the last row is the numbers from the 250GB system drive, which indicates the unraid hd is much faster than my zpool)

rewrite  seeks  ran_create  putc_latency  rewrite_latency  seeks_latency
11275    228246 2249        3770          116ms            191ms
10556    229326 7133        5388          147ms            247ms
10989    227938 13337       13569         128ms            141ms
11025    227938 873         3679          117ms            224ms
10926    229491 3580        6186          119ms            231ms
64389    111633 29779       30298         47190us          51692us

I also did time dd if=/dev/zero of=foo bs=1048576 count=1024 and got

1024+0 records in
1024+0 records out
1073741824 bytes (1.1GB) copied, 83.3947 s, 12.9 MB/s

real    1m23.397s
user    0m0.000s
sys     0m1.760s

Same command on the system drive is

1024+0 records in
1024+0 records out
1073741824 bytes (1.1GB) copied, 9.52386 s, 113 MB/s

real 0m9.752s
user 0m0.00s
sys  0m2.984s

It looks like my zpool is running very slow. Did I do anything wrong when I created my zpool? How can I tune it to make it perform the way it should be?

share|improve this question
up vote 1 down vote accepted

I suspect your hard-disk to report 512 bytes sectors while it really has 4 K ones. This is well known to have a very negative impact on performance, especially with RAID-Z configurations.

The workaround is to add the ashift=12 option at pool creation time.

Some details here

share|improve this answer
According to Grant Pannell 4k sectors were not that a problem:… – Mattias Åslund Mar 18 '12 at 7:20
This document still says "in some areas the ashift has a significant boost in performance". In any case, there have been a sufficiently large number of people reporting poor performance with these drives for the issue to be real. 4 K Partition alignment plays an important role too. – jlliagre Mar 18 '12 at 12:41
I seemed to remember I read the best practice guide and did use ashift=12 option. But I am not 100% sure. How can I check whether I did that or not? And if I didn't, do I have to recreate the pool? I already put a lot of data in it. – Martin Mar 18 '12 at 22:15
You can get this setting value by running zdb tank | grep ashift. It must be set at pool creation time. – jlliagre Mar 19 '12 at 6:16
If you run an iostat -xen while the bonnie is running, what's the output look like for asvc_t and wsvc_t on the pool disks? I note from your bonnie output that the latencies reported for the ZFS pool are 100+ms. That's just flat out pathetic. Something is very wrong. If wsvc_t is higher than like 0.3, the controller you're using is crap/overloaded; if asvc_t is getting over 20-30ms ever, your disks are bad or being talked to poorly, etc. Also since you have no ZIL device, for testing I'd suggest disabling ZIL mechanics.. – Nex7 Mar 20 '12 at 6:28

Well, your system drive is faster than your component drives, so you're not comparing apples to apples.

But that's certainly way lower than one would expect - keep in mind that the native ZFS on Linux project is not terribly mature yet, and not optimized for performance.

Is there other IO going on in your ZFS pool at the same time you're testing? Is the CPU saturating during the dd?

share|improve this answer
I think the system was in idle status when I did those test because I am the only one using the server and I wasn't doing anything. – Martin Mar 18 '12 at 22:11

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.