Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a Solaris 10 ZFS based NFS server. The physical disks are more or less at their maximum io rates. Performance is very bad - so we will add spindles. The NFS solely serves as storage for XenServer Hypervisors.

I want to know which VM disks (means .vhd files on the storage) are producing most of the load. How can I query the filesystem or maybe nfsd to get an iostat or top like output with filename and reads / writes. Numbers can be absolute or relative.

I tried iosnoop. It definitely goes in the right direction. Unfortunately it seems to be unable to get the filenames on a ZFS filesystem. I have no experience with dtrace. Maybe there is already a script out there?

share|improve this question

You need to use dtrace, for that level of introspection. Here's basically the nfsv3fileio.d example from, but updated to run live (I think, my testing was minimal). That same page has a few more examples. You might also Google for 'nfssvrtop'.

#!/usr/sbin/dtrace -s

#pragma D option quiet

        trace("Tracing.. hit CTRL-C to end. Updates every 5 seconds.\n");

        @bytes_read[args[1]->noi_curpath] = sum(args[2]->;

        @bytes_written[args[1]->noi_curpath] = sum(args[2]->res_u.ok.count);

        trunc(@bytes_read, 15);
        trunc(@bytes_written, 15);
        printf("\n%15s   %15s   %s\n", "Bytes Read/5s", "Bytes Written/5s", "Pathname");
        printa("%@15d   %@15d   %s\n", @bytes_read, @bytes_written);
share|improve this answer

Do you have any level of NVRAM write caching on your setup? In the ZFS case, it would be the presence of a ZIL device? If not, that's probably the key to your performance problems.

share|improve this answer
Actually the ZIL has gone corrupted (bad SSDs). That will be replaced asap. So you are right, no ZIL at the moment. – Roman Mar 15 '13 at 14:59
... And because "asap" is a bit too distant in time, I want to move some major load to another system. – Roman Mar 15 '13 at 15:06
I would disable ZIL for the moment, as that will help performance while you work to replace your ZIL SSDs. – ewwhite Mar 15 '13 at 17:04
Do you mean the ZFS wide zil? Which is now on the spindles? – Roman Mar 15 '13 at 22:39
Right, but you need to set sync=disabled for the relevant filesystems. – ewwhite Mar 15 '13 at 22:54

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.