Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a Sun T5220 server with the onboard LSI card and two disks that were in a RAID 1 mirror. The data is not important right now but we had a failed disk and are trying to understand how to do this for real if we had to recover from a failure.

The initial situation looked like this:

# raidctl -l c1t0d0
Volume                  Size    Stripe  Status   Cache  RAID
         Sub                     Size                    Level
                 Disk
----------------------------------------------------------------
c1t0d0                  136.6G  N/A     DEGRADED OFF    RAID1
                 0.1.0   136.6G          GOOD
                 N/A     136.6G          FAILED

Green light on the 0.0.0 disk. Find / lights up the 0.1.0 disk. So I know I have a bad drive and which one it is. Server still boots obviously.

First, we tried putting a new disk in. This disk came from an unknown source. Format would not see it, cfgadm -al would not see it so raidctl -l would not see it. I figure it's bad. We tried another disk from another spare server:

# raidctl -c c1t1d0 c1t0d0  (where t1 is my good disk - 0.1.0)
Disk has occupied space.

Also the different syntax options don't change anything:

# raidctl -C "0.1.0 0.0.0" -r 1 1
Disk has occupied space.

# raidctl -C "0.1.0 0.0.0" 1
Disk has occupied space.

Ok. Maybe this is because the disk from the spare server had a RAID 1 on it already. Aha, I can see another volume in raidctl:

# raidctl -l
Controller: 1
         Volume:c1t1d0  (this is my server's root mirror)
         Volume:c1t132d0  (this is the foreign root mirror)
         Disk: 0.0.0
         Disk: 0.1.0
         ...

No problem. I don't care about the data, I'll just delete the foreign mirror.

# raidctl -d c1t132d0
(warning about data deletion but it works)

At this point, /usr/bin/ binaries freak out. By that I mean, ls -l /usr/bin/which shows 1.4k but cat /usr/bin/which gives me a newline. Great, I just blew away the binaries (ie: binaries in mem still work)? I bounce the box. It all comes back fine. WTF. Anyway, back to recreating my mirror.

# raidctl -l
Controller: 1
         Volume:c1t1d0  (this is my server's root mirror)
         Disk: 0.0.0
         Disk: 0.1.0
         ...

Man says that you can delete a mirror and it will split it. Ok, I'll delete the root mirror.

# raidctl -d c1t0d0
Array in use.  (this might not be the exact error)

I googled this and found of course you can't do this (even with -f) while booted off the mirror. Ok. I boot cdrom -s and deleted the volume.

Now I have one disk that has a type of "LSI-Logical-Volume" on c1t1d0 (where my data is) and a brand new "Hitachi 146GB" on c1t0d0 (what I'm trying to mirror to):

(booted off the CD)
# raidctl -c c1t1d0 c1t0d0 (man says it's source destination for mirroring)
Illegal Array Layout.

# raidctl -C "0.1.0 0.0.0" -r 1 1  (alt syntax per man)
Illegal Array Layout.

# raidctl -C "0.1.0 0.0.0" 1  (assumes raid1, no help)
Illegal Array Layout.

Same size disks, same manufacturer but I did delete the volume instead of throwing in a blank disk and waiting for it to resync. Maybe this was a critical error. I tried selecting the type in format for my good disk to be a plain 146gb disk but it resets the partition table which I'm pretty sure would wipe the data (bad if this was production).

Am I boned? Anyone have experience with breaking and resyncing a mirror? There's nothing on Google about "Illegal Array Layout" so here's my contrib to the search gods.

share|improve this question

1 Answer 1

As it turns out, I could not find an answer to this. But I did find a workaround and some good information. First off, this was on 10/08 (U6) of Solaris 10. I booted off a 10/09 boot CD (U8) and found that there's a bug in raidctl on U8. U8 fails with a "Corrupt labe - bad geometry" error. Even when I wiped the disks completely I was unable to re-create a mirror using the U8 boot CD but on U7 (and presumably U6) the exact same command worked. So just a bit of version warning there.

The gist of the workaround went something like this (substitute your disks, paths, etc).

  • My partitions were split but I could see the data off the boot CD. I needed a lot of space to do a ufsdump so I imported a large zfs pool. This could mean different things to you, maybe just mount a large partition. Let's call it /mnt/space.
  • Copy or dump each of your existing partition table to a file. Remember you're booted off the CD at this point.
    • format (1, p, p) or do a prtvtoc /dev/rdsk/c1t0d0s2 > /mnt/space/partitions.txt
  • Backup partitions
    • ufsdump 0f - /dev/rdsk/c1t0d0s0 > /mnt/space/root_c1t1d0s0.dmp
    • ufsdump 0f - /dev/rdsk/c1t1d0s4 > /mnt/space/var_c1t1d0s4.dmp
    • continue for each partition
  • Recreate raid (off U7 or older CD, U8 has a bug, fails).
    • raidctl -c c1t0d0 c1t1d0 WARNING: This wipes both drives.
    • Creating RAID volume will destroy all data on spare space of member disks, proce ed (yes/no)? yes
  • Label new raid disk with format. You should not receive weird or failed labeling errors in format.
  • Lookup your volume with raidctl -l (assuming it's c1t1d0 in these instructions)
  • Restore your partition layout.
    • cat /mnt/space/partitions.txt | fmthard -s - /dev/rdsk/c1t1d0s2
  • At this point I actually switched to DiskSuite but the restore steps are similar.
  • newfs each of the partitions.
    • newfs /dev/rdsk/c1t1d0s0 (through s7, skip s2 obviously)
  • Mount and restore each partition:
    • mkdir /tmp/s0
    • mount /dev/rdsk/ctt1d0s0 /tmp/s0
    • cd /tmp/s0
    • cat /mnt/space/root_c1t1d0s0.dmp | ufsrestore xf - (answer yes to root dir permission)
    • umount /tmp/s0
    • repeat for each slice
  • Copy the boot block. The path to this file depends on your hardware:
    • installboot platform/SUNW,SPARC-Enterprise-T5220/lib/fs/ufs/bootblk /dev/rdsk/c1t1d0s0
  • Unmount everything, export zpools if need be, reboot off the CD.
  • Edit your alias from the open boot prompt
    • probe-scei-all
    • show-disks (select disk)
    • nvalias disk Ctrl-Y
    • boot disk
  • At this point you should be back to a hardware mirror or perhaps you switched to DiskSuite.

If you want to switch to disksuite in the middle of all this: - backup using ufsdump as above - delete your hardware raid definition - restore partitions to a 1st disk, newfs the partitions on the 1st disk - ufsrestore to the 1st disk, do a sanity boot - then start the regular disksuite install. If you try to do disksuite off the CD all in one-shot, it won't take because the meta service isn't running. You'll get this error: metadb: network/rpc/meta:default: failed to enable/disable SVM service Doing the meta commands won't hurt, they just won't stick. IE: when you reboot off your harddisk, metastat will says "no meta databases found".

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.