2

I've got a Nexenta/ZFS NAS that I'm using as an NFS backing store for a small VMware vSphere farm. At the moment I have 9x1TB disks all mirrored (last disk is configured as a write log device).

The disk performance is pretty good for my needs over NFS. However, one thing I've noticed is that if I do any IO-heavy operations on the NAS directly, VM performance slows to a crawl. An example would be copying 1TB of data between two different ZFS filesystems within the same zpool.

Is there any good way I can ensure that IO requests performed by the NFS daemon are prioritized over other IO operations on disk? In an ideal world, I'd have my VMs backing onto a completely separate zpool, so that they're unaffected by load on the ZFS filesystems. However, I'm wondering if there's a good way of doing it with a single zpool.

Linux has ionice, so if I were using that I could prefix a mv command with an ionice if I were to move a lot of data at a low IO priority. However, I don't think that's available on the Solaris kernel though.

Any suggestions?

Improve this question

4 Answers 4

Reset to default
4

AFAIK, to ZFS all I/O is I/O. What I mean by that is that it won't differentiate between your local operations and the ones NFS is asking ZFS to do.

You could play with the scheduling classes to somehow slow down your userland process that is copying all this data locally.

BTW, your dedicated 1TB disk for write log device won't help you at all, unless that specific disk is much faster than the rest (eg. SATA 7200 vs SAS 15k). We usually use SSDs for log/cache devices, or nothing at all.

Improve this answer
2
  • 1
    +1 A ZIL has to be on a faster disk or you get no performance boost. Also it will only use ~2x the RAM ZFS is using for ZIL. So a whole 1TB disk will have a lot of unused space.
    – Chris S
    Mar 14, 2011 at 14:16
  • TBH, I did have it as a hotspare, but thought I'd turn it into a log device to see if it made any difference. I'm seeing quite a lot of writes to the device itself, but don't really think performance has improved. I'll probably get another 3 1TB disks and use them + the spare to add another 2TB to the array.
    – growse
    Mar 14, 2011 at 14:21
2

Add more RAM.. after that. You might want to consider adding some more RAM. L2ARC's on SSD's are great, I have several. They don't need to be mirrored. The ZIL has to be mirrored (doesn't have to but its game over if its not and there is a some failure). You might want to add some RAM too.

Derek

Improve this answer
1

You won't have any way to throttle your I/O, unless you use some ugly concoction of pv and dd... but that's not worth it. The other poster is correct in that your 9th disk is useless as ZIL. An SSD or no disk is going to be better for that. It could be useful for L2arc cache, though.

Also see: http://opensolaris.org/jive/thread.jspa?messageID=493112

Improve this answer
1
  • That last disk would probably be pretty useless as a L2Arc unless it's fast again, as the mirror sets will have double the read speed of a single disk.
    – Chris S
    Mar 14, 2011 at 14:17
-1

Could you run your IO heavy tasks from yet-another-VM?

There will be a lot of overhead, yes, and you still have to make sure that you don't overload your NFS, but that way all traffic is treated equally.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .