Build and migrated to software raid (mdadm) on GPT disk, now can't assemble array

Question

mdadm, gpt issues, unrecognized partitions.

Simplified question: How do I get mdadm to recognize GPT partitions?

I have been attempting to convert/copy my Ubuntu 11.10 OS from a single drive to software raid 1. I have done similar in the past, but in this case, I was adding in a drive that has been configured for GPT and I tried to work with that without fully looking into the implications.

Currently, I have a non-booting mdadm RAID 1 array of /dev/md127 (the OS assigned that and it keeps picking up). I am booting off of live USB keys, currently System Rescue CD from sysresccd. While gdisk and parted can see all the partitions, most of the OS utilities do not, including mdadm. My main goal is just to make the raid array accessible so I can get pull the data and start fresh (without using GPT).

/dev/md127

/dev/sda
 /dev/sda1 <- GPT type partition
  /dev/sda1 <- exists within the GPT part, member of md127
  /dev/sda2 <- exists within the GPT part, empty

/dev/sdb
 /dev/sdb1 <- GPT type partition
  /dev/sdb1 <- exists within the GPT part, member of md127

History:

POINT A: The original OS was install on sda (actually /dev/sda6). I used a the Ubuntu live usb to add sdb. I got warning from fdisk about GPT so I used gdisk to create a raid partition (sdb1) and mdadm to create a raid1 mirror with a missing drive. I had many issues getting this working (including being unable to get grub to install) but I eventually got it to boot using grub on sda and /dev/md127 off of sdb. So at point A, I had copied my OS from sda6 to md127 on sdb. I then booted into a rescue mode and attempted to get a bootloader onto sdb, which failed. I then discovered my mistake: I had installed the raid onto sdb instead of sdb1, essentially overwriting the sdb1 partition.

POINT B: I now had two copies of my data- one on md127/sdb, and one on sda. I destroyed data on sda and created a new GPT table on sda. I then created sda1 for the raid array, and sda2 for a scratch partition. I added sda1 into the raid array and let it rebuild. md127 now covered /dev/sdb and /dev/sda1 as fully active and synced.

POINT C: I rebooted onto linux rescue again and was still able to access the raid array. I then removed /dev/sdb from the array and created /dev/sdb1 for the raid. I added sdb1 to the array and let it sync. I was able to mount and access /dev/md127 without issues. Once it completed, both /dev/sda1 and /dev/sdb1 were GPT partitions and actively syncing.

POINT D (current): I rebooted again to test if the array would boot and grub failed to load. I booted off of my live thumb drive and found that I can no longer assemble the raid array. mdadm doesn't see the required partitions.

--
root@freshdesk /root % uname -a
Linux freshdesk 3.0.24-std251-amd64 #2 SMP Sat Mar 17 12:08:55 UTC 2012 x86_64 AMD Athlon(tm) II X4 645 Processor AuthenticAMD GNU/Linux



===
/proc/partitions and parted look good:
root@freshdesk /root % cat /proc/partitions
major minor  #blocks  name

   7        0     301788 loop0
   8        0  976762584 sda
   8        1  732579840 sda1
   8        2  244181703 sda2
   8       16  732574584 sdb
   8       17  732573543 sdb1
   8       32    7876607 sdc
   8       33    7873349 sdc1


(parted) print all
Model: ATA ST31000528AS (scsi)
Disk /dev/sda: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size   File system  Name                Flags
 1      1049kB  750GB   750GB  ext4
 2      750GB   1000GB  250GB               Linux/Windows data


Model: ATA SAMSUNG HD753LJ (scsi)
Disk /dev/sdb: 750GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End    Size   File system  Name        Flags
 1      1049kB  750GB  750GB  ext4         Linux RAID  raid


Model: SanDisk SanDisk Cruzer (scsi)
Disk /dev/sdc: 8066MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      31.7kB  8062MB  8062MB  primary  fat32        boot, lba


===
# no sda2, and I double the sdb1 is the one shown in parted
root@freshdesk /root % blkid
/dev/loop0: TYPE="squashfs"
/dev/sda1: UUID="75dd6c2d-f0a8-4302-9da4-792cc7d72355" TYPE="ext4"
/dev/sdc1: LABEL="PENDRIVE" UUID="1102-3720" TYPE="vfat"
/dev/sdb1: UUID="2dd89f15-65bb-ff88-e368-bf24bd0fce41" TYPE="linux_raid_member"

root@freshdesk /root % mdadm -E /dev/sda1
mdadm: No md superblock detected on /dev/sda1.

# this is probably a result of me attempting to force the array up, putting superblocks on the GPT partition
root@freshdesk /root % mdadm -E /dev/sdb1
/dev/sdb1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 2dd89f15:65bbff88:e368bf24:bd0fce41
  Creation Time : Fri Mar 30 19:25:30 2012
     Raid Level : raid1
  Used Dev Size : 732568320 (698.63 GiB 750.15 GB)
     Array Size : 732568320 (698.63 GiB 750.15 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 127

    Update Time : Sat Mar 31 12:39:38 2012
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 1
  Spare Devices : 1
       Checksum : a7d038b3 - correct
         Events : 20195


      Number   Major   Minor   RaidDevice State
this     2       8       17        2      spare   /dev/sdb1

   0     0       8        1        0      active sync   /dev/sda1
   1     1       0        0        1      faulty removed
   2     2       8       17        2      spare   /dev/sdb1


===

root@freshdesk /root % mdadm -A /dev/md127 /dev/sda1 /dev/sdb1
mdadm: no recogniseable superblock on /dev/sda1
mdadm: /dev/sda1 has no superblock - assembly aborted
    root@freshdesk /root % mdadm -A /dev/md127  /dev/sdb1
mdadm: cannot open device /dev/sdb1: Device or resource busy
mdadm: /dev/sdb1 has no superblock - assembly aborted

Answer 1

mdadm doesn't recognize partitions, the Linux kernel does. A software RAID array doesn't need to know or care what type of partitions the disk uses, because it just uses the block devices that the kernel provides for the partitions. I'm using mdadm arrays on GPT disks on several computers and they work fine.

The partition layout you described doesn't make sense:

/dev/sda
 /dev/sda1 <- GPT type partition
  /dev/sda1 <- exists within the GPT part, member of md127
  /dev/sda2 <- exists within the GPT part, empty

/dev/sdb
 /dev/sdb1 <- GPT type partition
  /dev/sdb1 <- exists within the GPT part, member of md127

In particular, it looks like you're saying that sda2 is located within sda1. Partitions don't exist within other partitions, and GPT is a characteristic of the whole-disk device, not a partition. I think what you actually mean is:

/dev/sda <- GPT disk
 /dev/sda1 <- member of md127
 /dev/sda2 <- empty

/dev/sdb <- GPT disk
 /dev/sdb1 <- member of md127

However, your blkid output says that /dev/sda1 currently contains an Ext4 filesystem, not a RAID superblock — it's not a member of md127. It's not clear how that filesystem got there, since you said that you were using it as a RAID component, but since your story is long and full of twists, I suspect there may have been points where things happened that you didn't realize had happened. My suggestion at this point is:

• Assemble the array in degraded mode using just /dev/sdb1. Check that it contains your data; if not, check whether /dev/sda1 somehow contains an intact filesystem with your data, otherwise I hope you have a backup.
• Make a backup of all your data, if you don't have one already.
• Completely wipe /dev/sda: dd if=/dev/zero of=/dev/sda bs=1M. Then use gdisk to recreate the partition(s).
• Create a new degraded array using only a partition on sda. Make a filesystem in it, and copy your data into it.
• Disassemble the array that's using sdb1, and completely wipe /dev/sdb: dd if=/dev/zero of=/dev/sdb bs=1M. Then use gdisk to recreate the partition.
• Add /dev/sdb1 to the new array and let it sync.

As for installing GRUB, it depends on whether your machine supports EFI (and whether you're using it for booting). If you're using EFI, you need to make an EFI system partition somewhere; it should be roughly 100MB, formatted FAT32. Then you'd install the EFI version of GRUB. I won't go into too much detail on this; EFI booting is a topic for a separate question.

If you're not using EFI to boot, you need to make a "BIOS Boot" partition somewhere on the disk that you'll be installing GRUB on. (This is partition type code ef02 in gdisk.) The partition can be tiny; 1MB is plenty. GRUB will use this to store the boot code that it would have written to sectors 1 through 62 on an MBR disk. (On an MBR disk, those sectors are typically unallocated since the first partition typically begins at sector 63, but on a GPT disk, the partition table is located in that area.) GRUB should automatically notice that the disk you're installing it to contains a BIOS Boot partition, and put its boot code there instead of in sectors 1-62.