16

I am running an unmanaged Ubuntu server and due to recent performance issue and sent a request to the provider only to receive the following response:

"Dear Client,

as requested we have checked your drives. Both drives pass a test without any error and do not show any critical value here. We noticed the RAID is degraded state so you will need to re-add the drive to array here:

-----------------%<-----------------
Personalities : [raid1]
md3 : active raid1 sda4[0]
      1839089920 blocks super 1.2 [2/1] [U_]

md2 : active raid1 sda3[0]
      1073610560 blocks super 1.2 [2/1] [U_]

md1 : active raid1 sda2[0]
      523968 blocks super 1.2 [2/1] [U_]

md0 : active raid1 sda1[0]
      16768896 blocks super 1.2 [2/1] [U_]

unused devices: <none>
-----------------%<-----------------"

Google says that U_ indicates a failed drive, which appears to be in contradiction to what the support said. How do I proceed from here?

Something like

mdadm /dev/mdX -r /dev/sdaY  
mdadm /dev/mdX -a /dev/sdaY  

And if so, what is X and Y for my case? If more info is needed about my configuration, please let me know.

EDIT, more info below:

root@Ubuntu-1204-precise-64-minimal /home # df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/md2       1008G   26G  931G   3% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
udev             16G  4.0K   16G   1% /dev
tmpfs           3.2G  504K  3.2G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none             16G     0   16G   0% /run/shm
none            100M     0  100M   0% /run/user
/dev/md1        488M   66M  397M  15% /boot
/dev/md3        1.7T   31G  1.6T   2% /home

root@Ubuntu-1204-precise-64-minimal /home # mdadm -D /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Wed Nov  6 08:02:41 2013
     Raid Level : raid1
     Array Size : 16768896 (15.99 GiB 17.17 GB)
  Used Dev Size : 16768896 (15.99 GiB 17.17 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Wed Dec 21 03:50:13 2016
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : rescue:0
           UUID : 872ad258:c42ccb36:e9e19c96:98b55ee9
         Events : 816844

    Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1
       1       0        0        1      removed

root@Ubuntu-1204-precise-64-minimal /home # mdadm -D /dev/md1
/dev/md1:
        Version : 1.2
  Creation Time : Wed Nov  6 08:02:41 2013
     Raid Level : raid1
     Array Size : 523968 (511.77 MiB 536.54 MB)
  Used Dev Size : 523968 (511.77 MiB 536.54 MB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Wed Dec 21 07:30:55 2016
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : rescue:1
           UUID : 18cb39fc:9eaea61c:0074a6c2:661b5862
         Events : 387

    Number   Major   Minor   RaidDevice State
       0       8        2        0      active sync   /dev/sda2
       1       0        0        1      removed

root@Ubuntu-1204-precise-64-minimal /home # mdadm -D /dev/md2
/dev/md2:
        Version : 1.2
  Creation Time : Wed Nov  6 08:02:42 2013
     Raid Level : raid1
     Array Size : 1073610560 (1023.87 GiB 1099.38 GB)
  Used Dev Size : 1073610560 (1023.87 GiB 1099.38 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Wed Dec 21 07:58:52 2016
          State : active, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : rescue:2
           UUID : eb9be750:7ff778b4:31fd7ce9:9d86d191
         Events : 38799724

    Number   Major   Minor   RaidDevice State
       0       8        3        0      active sync   /dev/sda3
       1       0        0        1      removed

root@Ubuntu-1204-precise-64-minimal /home # mdadm -D /dev/md3
/dev/md3:
        Version : 1.2
  Creation Time : Wed Nov  6 08:02:42 2013
     Raid Level : raid1
     Array Size : 1839089920 (1753.89 GiB 1883.23 GB)
  Used Dev Size : 1839089920 (1753.89 GiB 1883.23 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent

    Update Time : Wed Dec 21 07:57:41 2016
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 0
  Spare Devices : 0

           Name : rescue:3
           UUID : c9b748ef:332d3bf9:5fa8fef1:5b433b0a
         Events : 7429895

    Number   Major   Minor   RaidDevice State
       0       8        4        0      active sync   /dev/sda4
       1       0        0        1      removed
4
  • Are you seeking a way to fix that drive problem, or how to deal with the support answer ?
    – Tolsadus
    Dec 21, 2016 at 7:00
  • Please note that I have reverted your spell correction (readd is really readd, or re-add, not read). Dec 21, 2016 at 7:02
  • 1
    If the controller ejected a drive from the array, even if no hard error is shown I never thrust back a disk. I already seen a drive without error repluged and it corrupted the array data.
    – yagmoth555
    Dec 21, 2016 at 12:19

3 Answers 3

18

Adding Back a RAID 1 Member

To add a RAID member back into the array, use:

mdadm <RAID Device> -a <Device to add into the array>

For example:

$ mdadm /dev/md0 -a /dev/sdc1
mdadm: re-added /dev/sdc1

If you check again using mdadm --detail , you'll see that /dev/sdc1 is shown as "rebuilding":

Number   Major   Minor   RaidDevice State
   0       8       17        0      active sync   /dev/sdb1
   2       8       33        1      spare rebuilding   /dev/sdc1

This means that the newly added member is being synchronised with the data on the other member; if you now check proc/mdstat, you can see the synchronisation progress - as a percentage and a bar chart:

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sdc1[2] sdb1[0]
      976759936 blocks [2/1] [U_]
      [>....................]  recovery =  1.2% (12409024/976759936) finish=195.2min speed=82312K/sec

unused devices: <none>

Verify that you can still see your files on the RAID drive following the addition of the new RAID member:

$ ls /mnt/raid1
Music  lost+found  Images Films Misc
3
2

I solved my own issue. I did not understand that mdX are partitions and sda and sdb are actual physical discs. So sdb was missing, so I needed to add it back. I executed the following commands:

sgdisk -R /dev/sdb /dev/sda
sgdisk -G /dev/sdb
mdadm /dev/md0 -a /dev/sdb1
mdadm /dev/md1 -a /dev/sdb2
mdadm /dev/md2 -a /dev/sdb3
mdadm /dev/md3 -a /dev/sdb4

After a day of resync (can be observed with)

cat /proc/mdstat

What I see today looks good again:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdb4[2] sda4[0]
      1839089920 blocks super 1.2 [2/2] [UU]

md1 : active raid1 sdb2[2] sda2[0]
      523968 blocks super 1.2 [2/2] [UU]

md2 : active raid1 sdb3[2] sda3[0]
      1073610560 blocks super 1.2 [2/2] [UU]

md0 : active raid1 sdb1[2] sda1[0]
      16768896 blocks super 1.2 [2/2] [UU]
1
  • Congratulations !
    – Tolsadus
    Dec 22, 2016 at 12:58
1

Before adding the drive as suggested by @Tolsadus its a good idea to check your drive health using smartmontools by executing short and long tests:

Initially you have to ensure that the hard disk supports SMART and is enabled:

smartctl -i /dev/sdb

short test:

smartctl -t short /dev/sdb

long test:

smartctl -l selftest /dev/sdb

3
  • I've seen this only now. Can I possibly run these tests AFTER I added sdb? Everything seems fine right now. Dec 22, 2016 at 13:06
  • Yes you can this after adding the disk Dec 22, 2016 at 13:12
  • I assume this is what their support team did to check the disks in the first place, but I agree that it is a good idea to run the tests again yourself.
    – Kevin
    Aug 5, 2020 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.