I've just built a small NAS/SAN box and am doing some testing with it. Configuration is:
- 2GB RAM
- 2x Opteron 265s
- Iwill DK8N
- 5x500GB SATA drives (1 spare, rest mirrored) = 1TB usable space
- Emulex LPC10000DC
- NexentaCore 3.0.2
The first thing I'm testing is COMSTAR performance over fibre-channel. I created a 700g zvol and exposed it over COMSTAR. Write-caching is enabled. My FC initiator is a Windows 7 box with a Q-Logic QLA2340 card in it.
Everything works great, except for write speeds. For some reason, I'm only seeing ~30MB/s, whereas I can peak at about 90MB/s read speeds, using IOMeter. Doing some hunting around, I ran a
zpool iostat command and saw the following pattern:
$ zpool iostat tank 1 tank 88.2G 840G 0 5.47K 0 43.4M tank 88.2G 840G 0 16.1K 0 128M tank 88.4G 840G 1 6.69K 8.93K 52.6M tank 88.4G 840G 52 4.89K 110K 38.3M tank 88.4G 840G 151 0 280K 0 tank 88.4G 840G 112 0 253K 0 tank 88.4G 840G 1 9.89K 16.0K 78.6M tank 88.4G 840G 0 14.3K 0 113M tank 88.6G 839G 150 2.99K 318K 20.2M tank 88.6G 839G 15 0 45.0K 0 tank 88.6G 839G 3 0 17.8K 0 tank 88.6G 839G 0 0 0 0 tank 88.6G 839G 0 0 0 0 tank 88.6G 839G 0 11.3K 0 89.9M tank 88.6G 839G 0 13.8K 0 110M tank 88.8G 839G 90 7.91K 184K 61.0M tank 88.9G 839G 105 91 220K 140K tank 88.9G 839G 42 0 84.6K 0 tank 88.9G 839G 0 0 0 0 tank 88.9G 839G 0 6.52K 7.99K 51.8M tank 88.9G 839G 0 15.6K 1011 124M tank 88.9G 839G 91 4.30K 193K 31.9M tank 89.1G 839G 1 9.48K 15.8K 74.7M tank 89.1G 839G 41 0 99.9K 0 tank 89.1G 839G 166 0 374K 0 tank 89.1G 839G 87 0 162K 0 tank 89.1G 839G 0 0 7.92K 0
It appears to be writing a lot, then stopping.
What's going on here?