Alright posting here in the hopes someone can help..
So I set up a 4x1.5TB RAID6 array. Got it built and everything. It worked fine. I copied over two more drives worth of Data, and then grew those two drives into the array. There was a couple hiccups but otherwise it worked fine and took forever.
I copied over the last drive worth of data today and followed the same steps to grow it into the array, and then of course did the "watch etc etc" command to keep a steady eye on the reshaping, because why not.
.... and it was going along fine, got to maybe 10-11% and I noticed it.. wasnt updating anymore.
This happened right as I was watching something from the array filesystem over the network.. then it suddenly locked.
I thought maybe something went hinky with the watch command, so I killed it, and just did "cat /proc/mdstat" from the cmdline.
and got nothing. No output or anything, the cursor goes down a line but nothing else happens.
If I try a "mdadm --detail /dev/md0", same thing. Nothing happens.
if i try to ls inside the mounted directory for the array, I get the root listing, but when I try to dig any deeper into folders, ls does the same thing as the first two commands and locks up, except I can't even ctrl-c out of it.
What I think is causing the issue, is I show like 7 smbd processes that are status D, which must be left over from when I was watching the video file and it locked.. of course I can't kill them, the system wont let me.
I am of course now incredibly paranoid that something went TOTALLY pear shaped, and I"m going to lose everything. I don't want to reboot because I have no idea if that will break anything.
edit: ok.. so:
I rebooted (I actually had to power off the machine. It would not die.)
I found this: www . linuxquestions .. org/questions/linux-server-73/raid-5-mdadm-grow-interrupted-what-to-do-next-602671/
which I tried, with no luck. I get
mdadm: superblock on /dev/sdb1 doesn't match others - assembly aborted
when I do mdadm -E on any of the drives, I get:
/dev/sdb1: Magic : a92b4efc Version : 00.91.00 UUID : 3c455e64:c4d0d230:c109596b:d7e29b7e Creation Time : Mon Nov 23 18:59:31 2009 Raid Level : raid6 Used Dev Size : 1465135936 (1397.26 GiB 1500.30 GB) Array Size : 7325679680 (6986.31 GiB 7501.50 GB) Raid Devices : 7 Total Devices : 7 Preferred Minor : 0
Reshape pos'n : 967965760 (923.12 GiB 991.20 GB) Delta Devices : 1 (6->7)
Update Time : Tue Dec 1 20:48:48 2009 State : active
Active Devices : 6 Working Devices : 6 Failed Devices : 1 Spare Devices : 0 Checksum : a4096474 - correct Events : 40943
Chunk Size : 64K
Number Major Minor RaidDevice State
this 5 8 17 5 active sync /dev/sdb1
0 0 8 49 0 active sync /dev/sdd1 1 1 8 65 1 active sync /dev/sde1 2 2 8 81 2 active sync /dev/sdf1 3 3 8 97 3 active sync /dev/sdg1 4 4 8 1 4 active sync /dev/sda1 5 5 8 17 5 active sync /dev/sdb1 6 6 0 0 6 faulty removed
That "faulty removed" is /dev/sdc1, which was the drive I just added and was trying to grow the array by.
Then I tried what I found here: http://ubuntuforums.org/showpost.php?p=2526794&postcount=2
sudo mdadm --create /dev/md0 --assume-clean --level=6 --raid-devices=7 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sda1 /dev/sdb1 missing
mdadm: /dev/sdd1 appears to be part of a raid array: level=raid6 devices=7 ctime=Mon Nov 23 18:59:31 2009 mdadm: /dev/sde1 appears to be part of a raid array: level=raid6 devices=7 ctime=Mon Nov 23 18:59:31 2009 mdadm: /dev/sdf1 appears to be part of a raid array: level=raid6 devices=7 ctime=Mon Nov 23 18:59:31 2009 mdadm: /dev/sdg1 appears to be part of a raid array: level=raid6 devices=7 ctime=Mon Nov 23 18:59:31 2009 mdadm: /dev/sda1 appears to contain an ext2fs file system size=1565523968K mtime=Mon Nov 30 23:59:03 2009 mdadm: /dev/sda1 appears to be part of a raid array: level=raid6 devices=7 ctime=Mon Nov 23 18:59:31 2009 mdadm: /dev/sdb1 appears to be part of a raid array: level=raid6 devices=7 ctime=Mon Nov 23 18:59:31 2009 Continue creating array? y mdadm: array /dev/md0 started.
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md0 : active raid6 sdb1 sda1 sdg1 sdf1 sde1 sdd1 7325679680 blocks level 6, 64k chunk, algorithm 2 [7/6] [UUUUUU_]
sudo lvdisplay raid
--- Logical volume --- LV Name /dev/raid/raid VG Name raid LV UUID R1fKIQ-dS9P-iOuN-BVGW-98Wo-Cc2w-rJ5wfp LV Write Access read/write LV Status available # open 0 LV Size 5.46 TB Current LE 5589 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 252:0
Hooray! everythings wonderful! NOT.
sudo mount /dev/raid/raid /blah
mount: wrong fs type, bad option, bad superblock on /dev/mapper/raid-raid, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so
augh if I had hair i'd be pulling it out. Help. I'd just assume not redownload 6TB of stuff :P