zfs with unsymmetric vdevs

Question

let's assume that i have a raidz created from three disks (or vdevs).

$ fallocate -l 64m /tmp/one
$ fallocate -l 64m /tmp/two
$ fallocate -l 64m /tmp/three
$ sudo zpool create test raidz /tmp/one /tmp/two /tmp/three
$ sudo zpool status test
  pool: test
 state: ONLINE
  scan: none requested
config:

    NAME            STATE     READ WRITE CKSUM
    test            ONLINE       0     0     0
      raidz1-0      ONLINE       0     0     0
        /tmp/one    ONLINE       0     0     0
        /tmp/two    ONLINE       0     0     0
        /tmp/three  ONLINE       0     0     0

errors: No known data errors

for the purpose of increasing the size of the pool, lets say that i have a set of 5 other disks - and i wanted to stripe the two together:

$ fallocate -l 64m four
$ fallocate -l 64m five
$ fallocate -l 64m six
$ fallocate -l 64m seven
$ fallocate -l 64m eight
$ sudo zpool add test raidz /tmp/four /tmp/five /tmp/six /tmp/seven /tmp/eight
invalid vdev specification
use '-f' to override the following errors:
mismatched replication level: pool uses 3-way raidz and new vdev uses 5-way raidz

hmm... so i can force it with -f:

$ sudo zpool add -f test raidz /tmp/four /tmp/five /tmp/six /tmp/seven /tmp/eight
$ sudo zpool status test
  pool: test
 state: ONLINE
  scan: none requested
config:

    NAME            STATE     READ WRITE CKSUM
    test            ONLINE       0     0     0
      raidz1-0      ONLINE       0     0     0
        /tmp/one    ONLINE       0     0     0
        /tmp/two    ONLINE       0     0     0
        /tmp/three  ONLINE       0     0     0
      raidz1-1      ONLINE       0     0     0
        /tmp/four   ONLINE       0     0     0
        /tmp/five   ONLINE       0     0     0
        /tmp/six    ONLINE       0     0     0
        /tmp/seven  ONLINE       0     0     0
        /tmp/eight  ONLINE       0     0     0

and everything 'looks' good...

why do i need to force it? what issues may i run into using this in production?

Answer 1

The reason zfs does not want you todo this, is that it will not be able to use all the space on the larger vdef.

Also, by composing the two vdevs redundancy will not be the same across all the data in the pool, which is a bad idea in any case.

On another level, NEVER use configurations that are actively discouraged with something like storage. As the 'bad side effects' may only show up once you are in production, and then all the help you will get is. Send/receive your data to a properly configured pool.