0

I'm checking a 4disk X 1TB raid5 array + cryptluks + lvm with 2partitions, / and /home. First I got a "not enough operational devices 2/4 failed" "Failed to RUN_ARRAY /dev/md/1: Input/output error. Kicking non-fresh /dev/sdc2.. kicking non-fresh /dev/sdb2 during boot.

After doing some things, I finally recovered my arrays and I can see my files using a livecd and mounting my devs. I started to see some "Failed to read /path/to/file during boot and my computer just frozed and did not read no file to boot my debian. Then ran smartctl long test with livecd and gave me a read failure in /dev/sdc so I changed with a new one (after marking /dev/sdc1 and /dev/sdc2 as failed + remove with mdadm --manage /dev/md0 --fail /dev/sdc1 --remove /dev/sdc1 && mdadm --manage /dev/md1 --fail /dev/sdc2 --remove /dev/sdc2 ), then I added /dev/sdc1 to md0 and /dev/sdc2 to md1, at this time md1 is rebuilding. Before I added the new hard disk I tried to boot my OS with 3 disks but still having same issue while booting my OS.

Here is my mdadm -D

/ (md0)

 Version : 1.2
 Creation Time : Mon Dec  8 23:40:34 2014
 Raid Level : raid5
 Array Size : 1460736 (1426.74 MiB 1495.79 MB)
 Used Dev Size : 486912 (475.58 MiB 498.60 MB)
 Raid Devices : 4
 Total Devices : 4
 Persistence : Superblock is persistent
 Update Time : Sat Feb  2 22:00:55 2019
 State : clean 
 Active Devices : 4
 Working Devices : 4
 Failed Devices : 0
 Spare Devices : 0
 Layout : left-symmetric
 Chunk Size : 512K
 Name : 0
 UUID : f7ee34dc:04067043:bbbc9df1:390b2f08
 Events : 741

Number   Major   Minor   RaidDevice State
   5       8       33        0      active sync   /dev/sdc1
   1       8        1        1      active sync   /dev/sda1
   2       8       49        2      active sync   /dev/sdd1
   4       8       17        3      active sync   /dev/sdb1

and /home (md1)

 Version : 1.2
 Creation Time : Fri Feb  1 23:35:23 2019
 Raid Level : raid5
 Array Size : 2928427008 (2792.77 GiB 2998.71 GB)
 Used Dev Size : 976142336 (930.92 GiB 999.57 GB)
 Raid Devices : 4
 Total Devices : 4
 Persistence : Superblock is persistent
 Intent Bitmap : Internal
 Update Time : Sat Feb  2 23:42:01 2019
 State : clean, degraded, recovering 
 Active Devices : 3
 Working Devices : 4
 Failed Devices : 0
 Spare Devices : 1
 Layout : left-symmetric
 Chunk Size : 512K
 Rebuild Status : 47% complete
 Name : 1
 UUID : dee3fe90:4d31dfc1:239f914a:b5f0fd94
 Events : 6154

Number   Major   Minor   RaidDevice State
   5       8       34        0      spare rebuilding   /dev/sdc2
   1       8        2        1      active sync   /dev/sda2
   2       8       50        2      active sync   /dev/sdd2
   4       8       18        3      active sync   /dev/sdb2

This is my /proc/mdstat

  md0 : active raid5 sdc1[5] sda1[1] sdb1[4] sdd1[2]
  1460736 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]

  md127 : active raid5 sdc2[5] sda2[1] sdb2[4] sdd2[2]
  2928427008 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [_UUU]
  [==========>..........]  recovery = 52.6% (514280288/976142336) finish=98.2min speed=78381K/sec
  bitmap: 0/8 pages [0KB], 65536KB chunk

unused devices: <none>

Now I have no errors with smartctl and waiting for the md1 rebuild

This is my mdadm -E

/dev/sda1:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0x0
  Array UUID : f7ee34dc:04067043:bbbc9df1:390b2f08
  Name : 0
  Creation Time : Mon Dec  8 23:40:34 2014
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 974336 (475.83 MiB 498.86 MB)
  Array Size : 1460736 (1426.74 MiB 1495.79 MB)
  Used Dev Size : 973824 (475.58 MiB 498.60 MB)
  Data Offset : 512 sectors
  Super Offset : 8 sectors
  Unused Space : before=432 sectors, after=512 sectors
  State : clean
  Device UUID : 343d3780:948d45c0:b0d48895:8fc387b0
  Update Time : Sat Feb  2 22:00:55 2019
  Checksum : f306a289 - correct
  Events : 741
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 1
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sda2:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0x1
  Array UUID : dee3fe90:4d31dfc1:239f914a:b5f0fd94
  Name : 1
  Creation Time : Fri Feb  1 23:35:23 2019
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 1952284672 (930.92 GiB 999.57 GB)
  Array Size : 2928427008 (2792.77 GiB 2998.71 GB)
  Data Offset : 262144 sectors
  Super Offset : 8 sectors
  Unused Space : before=262056 sectors, after=0 sectors
  State : clean
  Device UUID : 88e1fbd9:a8104c7c:289cd0bb:e2fe00d0
  Internal Bitmap : 8 sectors from superblock
  Update Time : Sat Feb  2 23:18:54 2019
  Bad Block Log : 512 entries available at offset 72 sectors
  Checksum : 5906c03e - correct
  Events : 5872
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 1
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdb1:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0x0
  Array UUID : f7ee34dc:04067043:bbbc9df1:390b2f08
  Name : 0
  Creation Time : Mon Dec  8 23:40:34 2014
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 974336 (475.83 MiB 498.86 MB)
  Array Size : 1460736 (1426.74 MiB 1495.79 MB)
  Used Dev Size : 973824 (475.58 MiB 498.60 MB)
  Data Offset : 512 sectors
  Super Offset : 8 sectors
  Unused Space : before=424 sectors, after=512 sectors
  State : clean
  Device UUID : fe9e5b46:5101684e:fdf3cd13:d2d1eeac
  Update Time : Sat Feb  2 22:00:55 2019
  Bad Block Log : 512 entries available at offset 72 sectors
  Checksum : c201a5ea - correct
  Events : 741
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 3
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdb2:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0x9
  Array UUID : dee3fe90:4d31dfc1:239f914a:b5f0fd94
  Name : 1
  Creation Time : Fri Feb  1 23:35:23 2019
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 1952284672 (930.92 GiB 999.57 GB)
  Array Size : 2928427008 (2792.77 GiB 2998.71 GB)
  Data Offset : 262144 sectors
  Super Offset : 8 sectors
  Unused Space : before=262056 sectors, after=0 sectors
  State : clean
  Device UUID : 7ebf7ff6:94303bff:eb2db9b7:ab8f1d65
  Internal Bitmap : 8 sectors from superblock
  Update Time : Sat Feb  2 23:19:24 2019
  Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
  Checksum : 897ee0db - correct
  Events : 5878
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 3
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdc1:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0x0
  Array UUID : f7ee34dc:04067043:bbbc9df1:390b2f08
  Name : 0
  Creation Time : Mon Dec  8 23:40:34 2014
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 974336 (475.83 MiB 498.86 MB)
  Array Size : 1460736 (1426.74 MiB 1495.79 MB)
  Used Dev Size : 973824 (475.58 MiB 498.60 MB)
  Data Offset : 512 sectors
  Super Offset : 8 sectors
  Unused Space : before=424 sectors, after=512 sectors
  State : clean
  Device UUID : 93699407:fa53c249:636ced08:81ba4897
  Update Time : Sat Feb  2 22:00:55 2019
  Bad Block Log : 512 entries available at offset 72 sectors
  Checksum : 5e0e243e - correct
  Events : 741
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 0
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdc2:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0xb
  Array UUID : dee3fe90:4d31dfc1:239f914a:b5f0fd94
  Name : 1
  Creation Time : Fri Feb  1 23:35:23 2019
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 1952284672 (930.92 GiB 999.57 GB)
  Array Size : 2928427008 (2792.77 GiB 2998.71 GB)
  Data Offset : 262144 sectors
  Super Offset : 8 sectors
  Recovery Offset : 713739888 sectors
  Unused Space : before=262056 sectors, after=0 sectors
  State : clean
  Device UUID : a1585481:ae213da1:db873970:c87c235c
  Internal Bitmap : 8 sectors from superblock
  Update Time : Sat Feb  2 23:19:39 2019
  Bad Block Log : 512 entries available at offset 72 sectors - bad blocks present.
  Checksum : 906680a9 - correct
  Events : 5881
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 0
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdd1:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0x0
  Array UUID : f7ee34dc:04067043:bbbc9df1:390b2f08
  Name : 0
  Creation Time : Mon Dec  8 23:40:34 2014
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 974336 (475.83 MiB 498.86 MB)
  Array Size : 1460736 (1426.74 MiB 1495.79 MB)
  Used Dev Size : 973824 (475.58 MiB 498.60 MB)
  Data Offset : 512 sectors
  Super Offset : 8 sectors
  Unused Space : before=432 sectors, after=512 sectors
  State : clean
  Device UUID : 043923f1:cd64f40b:e90fecf9:b594693c
  Update Time : Sat Feb  2 22:00:55 2019
  Checksum : 9fe681f2 - correct
  Events : 741
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 2
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdd2:

  Magic : a92b4efc
  Version : 1.2
  Feature Map : 0x1
  Array UUID : dee3fe90:4d31dfc1:239f914a:b5f0fd94
  Name : 1
  Creation Time : Fri Feb  1 23:35:23 2019
  Raid Level : raid5
  Raid Devices : 4
  Avail Dev Size : 1952284672 (930.92 GiB 999.57 GB)
  Array Size : 2928427008 (2792.77 GiB 2998.71 GB)
  Data Offset : 262144 sectors
  Super Offset : 8 sectors
  Unused Space : before=262056 sectors, after=0 sectors
  State : clean
  Device UUID : 7514baeb:f16719fc:77ffed9c:cd78d049
  Internal Bitmap : 8 sectors from superblock
  Update Time : Sat Feb  2 23:19:59 2019
  Bad Block Log : 512 entries available at offset 72 sectors
  Checksum : 457f27fd - correct
  Events : 5885
  Layout : left-symmetric
  Chunk Size : 512K
  Device Role : Active device 2
  Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
  • Is it possible that when rebuild finish I could not get this I/O error again while booting debian? or I will see it again?

  • I have no idea about what to do, I read something about running badblocks commands, according to mdadm --examine.. do I still have badblocks? is that right?

  • Not sure how safe is to run badblocks and not sure what arguments should I use. Found some things on Internet but was not clear for me.

I will appreciate any guidance in this, I started to know mdadm three days ago :/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.