Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I ran a test to measure my I/O IOPS on Linux:

iozone -s 4g -r 2k -r 4k -r 8k -r 16k -r 32k -O -b /tmp/results.xls

iozone claims that output is in operations per second yet the numbers are too big for that to be plausible. I'm observing some 320 CMDs/s maximum on vmware esx console (esxtop, then v).

    File size set to 4194304 KB
    Record Size 2 KB
    Record Size 4 KB
    Record Size 8 KB
    Record Size 16 KB
    Record Size 32 KB
    OPS Mode. Output is in operations per second.
    Command line used: iozone -s 4g -r 2k -r 4k -r 8k -r 16k -r 32k -O -b tmpresults.xls
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 Kbytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
                                                        random  random    bkwd   record   stride
          KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
     4194304       2   19025    5580    27581    29848     284     198     415  1103217     1498    18541     4340   24245    25618
     4194304       4   15650   21942    18962    21068     252    1198     193   976164     1677    22802    23093   21089    21232
     4194304       8   11121   11638    10273    10165     247    1196     202   625020^C

The test ran for 15 hours before I pressed ^C. Is that ordinary expectation for such command line (dedicated 4 drive RAID10 LUN, 10k RPM SAS drives in EMC CX300)?

share|improve this question
add comment

2 Answers

up vote 3 down vote accepted

It's strange that there doesn't seem to be a standard for IOPS measurement in Linux. These questions come up opten when people are asked to provide an estimate of IOPS for capacity planning or storage sizing. I end up using multiple tools to gain this info. First, you'll need to generate a load using the method of your choice; either a representative application load, or something like the iozone test you had above.

In another window, track IOPS using iostat -dxk 1, summing the r/s and w/s columns OR using IBM's nmon tool with the D option to track the Xfers column (which is essentially the sum of iostat's r/s and w/s).

Here's nmon output from a heavy sequential read using bonnie++ -u root -n 64:100000:16:64 (run from within the target directory). At this point, we're showing ~3200 IOPS while reading at about 258 Megabytes/second on a 6-disk RAID 1+0 array of 15k RPM SAS drives.

- Disk I/O -----(/proc/diskstats)------- all data is Kbytes per second ---------------------------------------------
|DiskName Busy    Read    Write       Xfers   Size  Peak%  Peak-RW    InFlight                                     |
|iss/c0d0 100% 264571.1    112.3KB/s 3238.7  81.0KB  100%  340584.5KB/s   2                                        |
|s/c0d0p1   0%      0.0      0.0KB/s    0.0   0.0KB    0%       2.0KB/s   0                                        |
|s/c0d0p2   0%      0.0      0.0KB/s    0.0   0.0KB   99%    1022.4KB/s   0                                        |
|s/c0d0p3   0%      0.0      0.0KB/s    0.0   0.0KB  100%    3636.5KB/s   0                                        |
|s/c0d0p4   0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0                                        |
|s/c0d0p5   0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0                                        |
|s/c0d0p6   0%      0.0      0.0KB/s    0.0   0.0KB    0%       0.0KB/s   0                                        |
|s/c0d0p7   0%      0.0     41.9KB/s    7.5   5.0KB  100%   16103.5KB/s   0                                        |
|s/c0d0p8   0%      0.0      0.0KB/s    0.0   0.0KB   79%     147.8KB/s   0                                        |
|s/c0d0p9 100% 264571.1     64.4KB/s 3230.2  81.0KB  100%  340538.5KB/s   2                                        |
--------------------------------------------------------------------------------------------------------------------
share|improve this answer
add comment

Basically you're getting a lot of help from caching and the sequential is fine but the random, especially writes seem low.

share|improve this answer
    
I'm I correct to assume that iozone tries to tell me that 2KB average random write is 198 IOPS? And that 4KB average random write is 1198 IOPS? –  Henno Feb 13 '11 at 14:07
    
That's your write cache working basically, it must have 4k chunks and anything less must make it very non-optimised. –  Chopper3 Feb 14 '11 at 10:22
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.