Mdadm won't create an array larger than 10TB

Question

• I have a server with six 4TB drives running Ubuntu 14.04 LTS.
• One of the drives is used for the system/OS
• I wanted to use the other 5 drives to get as close to 20TB of storage as possible
• It's a short term data analysis project, so RAID 0 is fine
• I used fdisk to create a single ~4TB partition on each of the 5 drives

After running mdadm to create a RAID 0 array, the result was just a 10TB device on /dev/md0. Half the size that I was expecting!

Here is the command that I used to create the array:

mdadm --create --verbose /dev/md0 --level=stripe --raid-devices=5 /dev/sd[bcdef]1

Below is an fdisk -l output. Any ideas on why I can't get more space out of the drives?

Disk /dev/sda: 4000.8 GB, 4000787030016 bytes
255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1  4294967295  2147483647+  ee  GPT

Disk /dev/sdb: 4000.8 GB, 4000787030016 bytes
90 heads, 3 sectors/track, 28940878 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xda4f04e2

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xc9e57616

Device Boot      Start         End      Blocks   Id  System
/dev/sdc1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sdd: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x53bed1ab

Device Boot      Start         End      Blocks   Id  System
/dev/sdd1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sde: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xbc5c1069

Device Boot      Start         End      Blocks   Id  System
/dev/sde1            2048  4294967294  2147482623+  83  Linux

Disk /dev/sdf: 4000.8 GB, 4000787030016 bytes
25 heads, 3 sectors/track, 104187162 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x4801480c

Device Boot      Start         End      Blocks   Id  System
/dev/sdf1            2048  4294967294  2147482623+  83  Linux

Disk /dev/md0: 10995.1 GB, 10995108413440 bytes
2 heads, 4 sectors/track, -1610614656 cylinders, total 21474821120 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 2621440 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Answer 1

Drop the 1 from the end of your command:

mdadm --create --verbose /dev/md0 --level=stripe --raid-devices=5 /dev/sd[bcdef]

Should work fine. I'm assuming you want to use all the space available on all 5 of the RAID disks.

Answer 2

fdisk will not create a partition bigger than 2TB (2^41 bytes). That seems to be pretty well documented; one example of many can be read here. 5*2TB=10TB, which is what you're getting.

You will need to use GNUparted to put a GPT partition table on each drive; then you'll be able to make a /dev/sdX1 partition that stretches all the way to the top of the drive.

Or, as jski suggests, you can use the raw devices instead of the partitions. I'd go for the partition approach, though, not least because you can always trim the partition to slightly below top-of-disc in case you ever need to replace a disc with a slightly smaller one (yes, I know that's a pointless consideration in RAID-0, but I feel it's a good habit to get into).