Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I had to open my file-server's housing on Sunday to replace a faulty fan. What I didn't see was that one of the sata-cables was not properly connected. The 1st thing I did after a reboot was a check of the RAID status and it showed immediately that one drive is missing. Till this moment the device was not used (however it was mounted, so I'm not 100% sure that system did nothing). I stopped md0 and re-plugged the cable:

mdadm --stop /dev/md0

After another reboot I checked the removed drive:

 mdadm --examine /dev/sdd1
       Checksum : 3276bc1d - correct
         Events : 315782

         Layout : left-symmetric
     Chunk Size : 32K

      Number   Major   Minor   RaidDevice State
this     0       8       49        0      active sync   /dev/sdd1

   0     0       8       49        0      active sync   /dev/sdd1
   1     1       8       65        1      active sync   /dev/sde1
   2     2       8       33        2      active sync   /dev/sdc1
   3     3       8       17        3      active sync   /dev/sdb1

I was a bit surprised that it was shown as active (even if earlier mdadm said, that this device was removed from array) and its checksum was OK. I recreated RAID with:

mdadm --assemble /dev/md0 --scan

The command mdadm --detail /dev/md0 showed that all drives were running and system was in "clean" state. I mounted the device md0 and then came hic-cup. I wanted to work on one of the last files that I had been using before all the situation and it was not there. In another place I missed actually all files from the directory where I was working. As far as I can see most of the files that are older than a few days are intact but some newer ones are missing.

Now the big question: what would be your advice? Is there a way to get these data? I thought about removing the drive that was earlier labeled by mdadm and rebuild array with another empty HDD.

I've found that after re-assemble the "broken" drive is on another place in RAID (before it was listed on 3rd place, then on 4th). Can this have influence on rebuilt process? If yes, how to reassemble the array properly? I'm sure the SATA-cables are connected still in the same order to the controller.

p.s. Please no advises like "restore from backup". I'm doing back-ups on Sunday's night and this happened in the late afternoon, so backup is not really options for me.

p.s.s. I asked this question on Unix&Linux but no answer came up during last two days. I'm getting quite anxious. Sorry for duplicating if any of you is reading the other forum.

share|improve this question
up vote 1 down vote accepted

Maybe it will help somebody.

I didn't write it before but all four partitions had the same count of events

mdadm --examine /dev/sd[a-z]1 | egrep 'Event|/dev/sd`'
mdadm: No md superblock detected on /dev/sda1.
         Events : 315786
         Events : 315786
         Events : 315784
         Events : 315786

Still, after some reading I decided to remove the "failed" drive and re-assamble my md0 device.

mdadm --manage --set-faulty /dev/md0 /dev/sdd1
mdadm /dev/md1 --stop
mdadm --assemble /dev/md0 /dev/sd[bce]1 --force 
mdadm --manage /dev/md0 --add /dev/sdd1

Please, don't ask me why it worked. The important part for me is that I got back all the files (file allocation table shows proper content of directories. All missing files are there.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.