Where do I find statistics about how IO is divided between zfs datasets? (zpool iostat only tells me how much IO a pool is experiencing.)

All the relevant datasets are used through NFS, so I'd be happy with per export NFS IO statistics also.

We're currently running OpenIndiana


It seems that operation and byte counter are available in kstat

kstat -p unix:*:vopstats_???????
unix:0:vopstats_2d90002:nputpage        50
unix:0:vopstats_2d90002:nread   12390785
unix:0:vopstats_2d90002:read_bytes      22272845340
unix:0:vopstats_2d90002:readdir_bytes   477996168

...but the strange hexadecimal ID numbers have to be resolved from /etc/mnttab (better ideas?)

rpool/export/home/jkj /export/home/jkj zfs rw,...,dev=2d90002 1308471917

Now writing a munin plugin to use the data...

  • 1
    Maybe with some dtrace spells? Also looked a bit in kstat, but didn't find anything per dataset. – jkj Jun 9 '11 at 11:49
  • From what I know, this information is available in the Fishworks interface for any Oracle ZFS appliances using DTRACE. So, the information is definitely there, somewhere. – Stefan Lasiewski Jun 9 '11 at 21:57

How about this... In this instance I have a ZFS filesystem '/volumes/lab3_pool_a/data02/alpha/' and I am using fsstat to get some basic statistics about it. Notice, I am getting stats for both, ZFS as a whole, and this one filesystem. By the way, fsstat has some good options and may give you just what you need.

# fsstat zfs /volumes/lab3_pool_a/data02/alpha/ 1

 new  name   name  attr  attr lookup rddir  read read  write write
 file remov  chng   get   set    ops   ops   ops bytes   ops bytes
    0     0     0    27     0     27     0     0     0     1   208 zfs
    0     0     0     0     0      0     0     0     0     0     0 /volumes/lab3_pool_a/data02/alpha
  • 3
    I've done something similar to this, but remember that fsstat operates at the VFS layer, so not all the IO you see here is actually going to/from your zpool. The VFS stats will include IO that is served from RAM cache as well. Additionally, ZFS reorders and coalesces quite a bit of IO, so when you compare fsstat to 'zpool iostat' you often see very different numbers. – eirescot Jun 9 '11 at 21:23
  • Yep, these are somewhat sanitized stats, but they may still be suitable, depending upon what the real need is for this data. – slashdot Jun 10 '11 at 1:12

Adding to slashdot's comments above: this one-liner does the same thing over all ZFS datasets (as determined by the pattern of the device column in mount(1M) output:

fsstat -i `mount | awk '{if($3 ~ /^[^\/:]+\//) {print $1;}}'` 1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.