Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am seeing the following under /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sda2[0](F) sdb2[1]
976554876 blocks super 1.1 [2/1] [_U]
bitmap: 4/8 pages [16KB], 65536KB chunk

md0 : active raid1 sda1[0](F) sdb1[1]
204788 blocks super 1.0 [2/1] [_U]

unused devices: <none>

It appears /dev/sda has failed. However, raid1 is redundant so my system should still be able to operate. This however, is not the case, my entire file system has been kicked into Read-only mode.

Running mdadm --detail gives me:
sudo: Can't open /var/run/sudo/supertrader/18: Read-only file system
/dev/md0:
        Version : 1.0
  Creation Time : Tue Aug 16 18:04:44 2011
     Raid Level : raid1
     Array Size : 204788 (200.02 MiB 209.70 MB)
  Used Dev Size : 204788 (200.02 MiB 209.70 MB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Thu Mar  1 09:48:13 2012
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 1
  Spare Devices : 0

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       17        1      active sync

       0       8        1        -      faulty spare

At this point, I would like to get the machine operating again (filesystem not locked in read-only mode) while I get somebody at the datacentre to replace the failed drive.

When I tried removing the failed drive using:

      mdadm --manage /dev/md0 --remove /dev/sda2

I get an error message about /dev/sda2 not found.

My question is the following: (1) What happens if I reboot the system now, will it automatically come up without the failed disk? Or do I need to find some way to actually remove the disk from the software array. I'm not local to the machine so if it gets stuck while booting, it is a real pain to fix.

Thanks in advance.

EDIT2: I went ahead with the reboot and the system came up again, this time with the filesystem mounted rw

I am not sure what happened, but it looks like the 'failed' disk came back online? here is the /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sdb2[1]
      976554876 blocks super 1.1 [2/1] [_U]
      bitmap: 5/8 pages [20KB], 65536KB chunk

md0 : active raid1 sda1[0] sdb1[1]
      204788 blocks super 1.0 [2/2] [UU]

unused devices: <none>

Prior to this, I had run:

mdadm /dev/md0 -r --force detached

I did this on both md0 and md1. md0 now looks normal, any idea what is happening with md1?

EDIT: Also, I noticed that under /dev/ both sda and sdb are missing, in fact, I only have sdc1 and sdc2.

What is going on here, why are both of my raid disks missing?

fdisk -l output:

Disk /dev/md0: 209 MB, 209702912 bytes 2 heads, 4 sectors/track, 51197 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000

Device Boot      Start         End      Blocks   Id  System

Disk /dev/md1: 1000.0 GB, 999992193024 bytes 2 heads, 4 sectors/track, 244138719 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table

Disk /dev/dm-0: 27.9 GB, 27917287424 bytes 255 heads, 63 sectors/track, 3394 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000

Disk /dev/dm-0 doesn't contain a valid partition table

Disk /dev/dm-1: 972.1 GB, 972071895040 bytes 255 heads, 63 sectors/track, 118181 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000

Disk /dev/dm-1 doesn't contain a valid partition table

Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0009984e

Device Boot Start End Blocks Id System /dev/sdc1 * 1 26 204800 fd Linux raid autodetect Partition 1 does not end on cylinder boundary. /dev/sdc2 26 121602 976556032 fd Linux raid autodetect

Is there a sure way to find out which is my surviving disk hardware wise?

share|improve this question
1  
Is the system remote did you install the bootloader to both disks? –  Zoredache Mar 1 '12 at 16:14
    
does executing partprobe make your machine recognize again sda and sdb? –  Carlos Campderrós Mar 1 '12 at 16:53
    
I did a standard mdadm raid1 with LVM, I'm not sure if that puts the bootloaded on both disks. –  user788171 Mar 1 '12 at 17:11
    
also, running partprobe ends up with failed to re-read the partition table, device or resource busy. however, i do get this message for sda, sdb, and md0 –  user788171 Mar 1 '12 at 17:12
    
finally, i did go ahead and reboot the system and now my raid is mounted as rw again. i edited my original post above and put the new /proc/mdstat message –  user788171 Mar 1 '12 at 17:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.