Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I'm trying to recover whatever data I can from a RAID5 array of a failed NAS.

The disks were physically moved to another server that so they could be imaged, but Knoppix "helpfully" detected and recovered the array on boot:-

[  677.791667] md: md0 stopped.
[  677.793443] md: bind<sdb>
[  677.793580] md: bind<sdc>
[  677.793866] md: bind<sdd>
[  677.794004] md: bind<sda>
[  678.671014] md: raid6 personality registered for level 6
[  678.671017] md: raid5 personality registered for level 5
[  678.671019] md: raid4 personality registered for level 4
[  678.671446] md/raid:md0: device sda operational as raid disk 0
[  678.671449] md/raid:md0: device sdc operational as raid disk 2
[  678.671452] md/raid:md0: device sdb operational as raid disk 1
[  678.671840] md/raid:md0: allocated 4222kB
[  678.671874] md/raid:md0: raid level 5 active with 3 out of 4 devices, algorithm 2
[  678.671877] RAID conf printout:
[  678.671879]  --- level:5 rd:4 wd:3
[  678.671881]  disk 0, o:1, dev:sda
[  678.671883]  disk 1, o:1, dev:sdb
[  678.671886]  disk 2, o:1, dev:sdc
[  678.671888]  disk 3, o:1, dev:sdd
[  678.671923] md0: detected capacity change from 0 to 4500901330944
  ^ RAID conf printout repeated
  ^ RAID conf printout repeated
[  678.679071] md: recovery of RAID array md0
[  678.679074] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
[  678.679077] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
[  678.679083] md: using 128k window, over a total of 1465137152 blocks.
[  678.679086] md: resuming recovery of md0 from checkpoint.
[  678.694387]  md0: unknown partition table
[18209.608703] md: md0: recovery done
[18209.690600] RAID conf printout:
[18209.690605]  --- level:5 rd:4 wd:4
[18209.690607]  disk 0, o:1, dev:sda
[18209.690610]  disk 1, o:1, dev:sdb
[18209.690612]  disk 2, o:1, dev:sdc
[18209.690614]  disk 3, o:1, dev:sdd

Afterwards, the array looks healthy, with /proc/mdstat showing:-

Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sda[0] sdd[4] sdc[2] sdb[1]
      4395411456 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]

However, I'm unable to mount /dev/md0 directly or find any logical volumes.

Using dd to extract the first 512 bytes of /dev/md0 gets me the partial contents of a file that was on the array instead of the MBR, so I suspect this is because the array was assembled in the wrong order.

If correct, how can I re-assemble the array?

If not, what is the best way to try and recover data from the array?


mdadm -D /dev/md0

/dev/md0:
        Version : 1.2
  Creation Time : Sun Aug  7 17:25:36 2011
     Raid Level : raid5
     Array Size : 4395411456 (4191.79 GiB 4500.90 GB)
  Used Dev Size : 1465137152 (1397.26 GiB 1500.30 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Fri Aug 12 21:54:53 2011
          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : Microknoppix:1  (local to host Microknoppix)
           UUID : 37b79cab:d800da6a:eec72e0d:6547b6ef
         Events : 19

    Number   Major   Minor   RaidDevice State
       0       8        0        0      active sync   /dev/sda
       1       8       16        1      active sync   /dev/sdb
       2       8       32        2      active sync   /dev/sdc
       4       8       48        3      active sync   /dev/sdd

mdadm -E /dev/sd?

/dev/sda:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 37b79cab:d800da6a:eec72e0d:6547b6ef
           Name : Microknoppix:1  (local to host Microknoppix)
  Creation Time : Sun Aug  7 17:25:36 2011
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 2930275120 (1397.26 GiB 1500.30 GB)
     Array Size : 8790822912 (4191.79 GiB 4500.90 GB)
  Used Dev Size : 2930274304 (1397.26 GiB 1500.30 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 42c9006b:1405120d:edf4d6da:e707f2dd

    Update Time : Mon Aug 15 02:19:35 2011
       Checksum : 40b694de - correct
         Events : 19

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)
/dev/sdb:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 37b79cab:d800da6a:eec72e0d:6547b6ef
           Name : Microknoppix:1  (local to host Microknoppix)
  Creation Time : Sun Aug  7 17:25:36 2011
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 2930275120 (1397.26 GiB 1500.30 GB)
     Array Size : 8790822912 (4191.79 GiB 4500.90 GB)
  Used Dev Size : 2930274304 (1397.26 GiB 1500.30 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 39fc5928:11ecb5b6:c3c088e6:2833b7ea

    Update Time : Mon Aug 15 02:19:35 2011
       Checksum : c02aa5ea - correct
         Events : 19

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AAAA ('A' == active, '.' == missing)
/dev/sdc:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 37b79cab:d800da6a:eec72e0d:6547b6ef
           Name : Microknoppix:1  (local to host Microknoppix)
  Creation Time : Sun Aug  7 17:25:36 2011
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 2930275120 (1397.26 GiB 1500.30 GB)
     Array Size : 8790822912 (4191.79 GiB 4500.90 GB)
  Used Dev Size : 2930274304 (1397.26 GiB 1500.30 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 90c64aa7:82b06090:fe9363e1:380dc0da

    Update Time : Mon Aug 15 02:19:35 2011
       Checksum : 3a9e1ff - correct
         Events : 19

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : AAAA ('A' == active, '.' == missing)
/dev/sdd:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 37b79cab:d800da6a:eec72e0d:6547b6ef
           Name : Microknoppix:1  (local to host Microknoppix)
  Creation Time : Sun Aug  7 17:25:36 2011
     Raid Level : raid5
   Raid Devices : 4

 Avail Dev Size : 2930275120 (1397.26 GiB 1500.30 GB)
     Array Size : 8790822912 (4191.79 GiB 4500.90 GB)
  Used Dev Size : 2930274304 (1397.26 GiB 1500.30 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : fab3ec51:50f9bbc6:7ac51700:54195903

    Update Time : Mon Aug 15 02:19:35 2011
       Checksum : 2bf455cf - correct
         Events : 19

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : AAAA ('A' == active, '.' == missing)

fdisk -l

Disk /dev/sda: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x894e894e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1      267350  2147483647+  ee  GPT

Disk /dev/sdb: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/md0: 4500.9 GB, 4500901330944 bytes
2 heads, 4 sectors/track, 1098852864 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 1572864 bytes
Disk identifier: 0x312e312f

Disk /dev/md0 doesn't contain a valid partition table
share|improve this question
    
Between this two timestamps, it did a recovery: [ 678.694387] md0: unknown partition table [18209.608703] md: md0: recovery done, which means it must have copied data from what it decided were good drives to the rest. I imagine it is possible it chose wrong. Were all 4 disks the same ones as in the original NAS? Is it possible that the NAS was using LVM, and if so, perhaps running a pvscan would help? –  Paul Aug 15 '11 at 10:10
    
All four were from the original NAS, but as I didn't move the drive, I cannot be sure if they were kept in order. When I couldn't find a filesystem on md0, LVM was my next thought, but pvscan, vgcsan and lvscan all came back with nothing (well, "no volume groups found", etc.). –  Kanji Aug 15 '11 at 19:18

1 Answer 1

I find it quite hard to believe that mdadm would assemble drives in the wrong order; it's metadata isn't that bad. I'm wondering what should be on the RAID array, and what dmesg had to say about matters when you tried (and failed) to mount the array.

If your surmise is correct about the array actually being out-of-order is correct, I'd unplug the disks and either give up on the data, or fork out money for a professional data-recovery place to try and put it back together. There's a chance you'll be able to do it, but there's also a chance that you'll make a mess of it and make the data harder to recover for the pros, and what is your time worth to basically learn data recovery on the fly?

For your pennance, say 10 "RAID is not a backup"s and 5 "Our Linus"es...

share|improve this answer
    
Completely agree that RAID is not a backup, but I inherited the array after it had failed and am just trying to make the best of a bad situation. I'm very aware that it may be beyond my abilities to recover anything useful at this point. re: mount, dmesg reports some variation on <type> (md0) Can't find <type> filesystem for each filesystem I tried (e.g., EXT4-fs (md0): VFS: Can't find ext4 filesystem) -- not at all surprising where (as I mentioned in the OP) I found partial contents of user data in the first 512b instead of partition info. –  Kanji Aug 15 '11 at 19:44
    
You wouldn't expect to find partition info on a block device that isn't partitioned. You're definitely out of your depth here; call in the professionals. –  womble Aug 15 '11 at 21:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.