I am encountering some problems with my RAID-5 array, using mdadm and Debian.
First of all, I lost a drive (completely, it is not even recognized by the BIOS), then I replaced it with a new one; rebuilding has started but has been interrupted by a read error on the second disk (and this one has been removed):
raid5:md0: read error not correctable (sector 1398118536 on sdd)
I suppose that this one will die in the next few days, but I want to re-add it to work with a degraded array to perform some backups (only few sectors are broken and I hope to save a maximum of data before it fails).
Here are my disks, in the RAID order:
- sdc - OK
- sdd - (the one with read errors, removed from array while rebuilding)
- sde - (the one that died, replaced by a spare, but obviously interrupted while rebuilding => I'm not convinced about its data integrity)
- sdf - OK
The fact is that I can't re-add sdd to the array, using this command:
# mdadm --assemble /dev/md0 /dev/sdc1 /dev/sdd1 /dev/sdf1 --force --run
mdadm: failed to RUN_ARRAY /dev/md0: Input/output error
mdadm: Not enough devices to start the array.
# mdadm -D /dev/md0
/dev/md0:
Version : 0.90
Creation Time : Tue Aug 24 14:20:39 2010
Raid Level : raid5
Used Dev Size : 1465039488 (1397.17 GiB 1500.20 GB)
Raid Devices : 4
Total Devices : 3
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Sun Oct 23 01:57:22 2011
State : active, FAILED, Not Started
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 128K
UUID : 01017848:84926c43:1751c931:a76e1cde (local to host tryphon)
Events : 0.131544
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 0 0 1 removed
2 0 0 2 removed
3 8 81 3 active sync /dev/sdf1
4 8 49 - spare /dev/sdd1
As you can see, sdd is recognized as a spare instead of being in sync as RAID device #1.
And I have no idea how to tell mdadm that sdd is RAID device #1.
If anyone has any idea, it would be so great!
Thanks.
PS: if this helps, here is the output of mdadm disk examination:
# mdadm -E /dev/sd[cdef]1
/dev/sdc1:
Magic : a92b4efc
Version : 0.90.00
UUID : 01017848:84926c43:1751c931:a76e1cde (local to host tryphon)
Creation Time : Tue Aug 24 14:20:39 2010
Raid Level : raid5
Used Dev Size : 1465039488 (1397.17 GiB 1500.20 GB)
Array Size : 4395118464 (4191.51 GiB 4500.60 GB)
Raid Devices : 4
Total Devices : 3
Preferred Minor : 0
Update Time : Sun Oct 23 01:57:22 2011
State : clean
Active Devices : 2
Working Devices : 3
Failed Devices : 2
Spare Devices : 1
Checksum : dfeeeace - correct
Events : 131544
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 0 8 33 0 active sync /dev/sdc1
0 0 8 33 0 active sync /dev/sdc1
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 81 3 active sync /dev/sdf1
4 4 8 49 4 spare /dev/sdd1
/dev/sdd1:
Magic : a92b4efc
Version : 0.90.00
UUID : 01017848:84926c43:1751c931:a76e1cde (local to host tryphon)
Creation Time : Tue Aug 24 14:20:39 2010
Raid Level : raid5
Used Dev Size : 1465039488 (1397.17 GiB 1500.20 GB)
Array Size : 4395118464 (4191.51 GiB 4500.60 GB)
Raid Devices : 4
Total Devices : 3
Preferred Minor : 0
Update Time : Sun Oct 23 01:57:22 2011
State : clean
Active Devices : 2
Working Devices : 3
Failed Devices : 2
Spare Devices : 1
Checksum : dfeeeae0 - correct
Events : 131544
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 4 8 49 4 spare /dev/sdd1
0 0 8 33 0 active sync /dev/sdc1
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 81 3 active sync /dev/sdf1
4 4 8 49 4 spare /dev/sdd1
/dev/sde1:
Magic : a92b4efc
Version : 0.90.00
UUID : 01017848:84926c43:1751c931:a76e1cde (local to host tryphon)
Creation Time : Tue Aug 24 14:20:39 2010
Raid Level : raid5
Used Dev Size : 1465039488 (1397.17 GiB 1500.20 GB)
Array Size : 4395118464 (4191.51 GiB 4500.60 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Update Time : Sat Oct 22 22:11:52 2011
State : clean
Active Devices : 2
Working Devices : 3
Failed Devices : 2
Spare Devices : 1
Checksum : dfeeb657 - correct
Events : 131534
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 4 8 65 4 spare /dev/sde1
0 0 8 33 0 active sync /dev/sdc1
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 81 3 active sync /dev/sdf1
4 4 8 65 4 spare /dev/sde1
/dev/sdf1:
Magic : a92b4efc
Version : 0.90.00
UUID : 01017848:84926c43:1751c931:a76e1cde (local to host tryphon)
Creation Time : Tue Aug 24 14:20:39 2010
Raid Level : raid5
Used Dev Size : 1465039488 (1397.17 GiB 1500.20 GB)
Array Size : 4395118464 (4191.51 GiB 4500.60 GB)
Raid Devices : 4
Total Devices : 3
Preferred Minor : 0
Update Time : Sun Oct 23 01:57:22 2011
State : clean
Active Devices : 2
Working Devices : 3
Failed Devices : 2
Spare Devices : 1
Checksum : dfeeeb04 - correct
Events : 131544
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 3 8 81 3 active sync /dev/sdf1
0 0 8 33 0 active sync /dev/sdc1
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 81 3 active sync /dev/sdf1
4 4 8 49 4 spare /dev/sdd1