6

I have three disks which used to hold an MD RAID5 array. I have since removed (or so I thought) this array and created partions for btrfs and swap space. On rebooting the machine, MD still binds the devices that used to hold the old array, causing the new filesystem to fail to mount.

It was suggested to me that the old superblocks of the raid arrays might be left behind causing MD to think it is a real array and thus binding the disks. The suggested solution was to use mdadm --zero-superblock to clear the superblock on the affected disks. However, I don't really know what this does with the disk. Since this disk holds partitions I don't really want to start zeroing parts of it blindly.

So what procedure should I follow to safely clear the MD superblocks without damaging the other partitions and file systems on the drives?

This question essentially asks the same thing, but there isn't a clear answer as to whether doing mdadm --zero-superblock on a repartitioned device is actually supposed to be safe: mdadm superblock hiding/shadowing partition

  • please query disk devices, check which version of superblock metadata is used – GioMac Aug 30 '13 at 21:51
  • @GioMac, version 0.90. I have done some reading and figured it out. I'll write it up in a moment. I don't think I can give a good general answer though so feel free to add yours. I probably won't accept mine as the correct one - it's more specific that I had hoped the answer would be :) – Simon Lindgren Aug 30 '13 at 22:49
6

https://raid.wiki.kernel.org/index.php/RAID_superblock_formats

The superblock is 4K long and is written into a 64K aligned block that starts at least 64K and less than 128K from the end of the device (i.e. to get the address of the superblock round the size of the device down to a multiple of 64K and then subtract 64K). The available size of each device is the amount of space before the super block, so between 64K and 128K is lost when a device in incorporated into an MD array.

So, It's already too late and might be unsafe to use --zero-superblock, because we don't know is there any data or not - you must resize/shrink your current partition to -128K from the end of the x-RAID partition, then, wipe that part and grow partition back.

Other option 1: write large files to fill entire disk, it will overwrite RAID superblocks and it will not be recognizable by the mdadm.

Other option 2: similar to 1: https://unix.stackexchange.com/questions/44234/clear-unused-space-with-zeros-ext3-ext4

  • So if overwriting the superblock with zeroes causes data loss, that means the superblock is at the same time a valid part of a filesystem structure, which in turn means specially crafted data can cause MD to find a spurious superblock. Correct? My point is that if --zero-superblock is unsafe, then the newer file system is indeed already damaged and in that case it doesn't matter anyway. – Simon Lindgren Aug 31 '13 at 0:18
  • Newer file system is created after RAID, so, it can't be damaged unless you mount it as RAID in RW mode again. But if you will erase RAID blocks - you don't know what part of newer file system will be erased with it, even few bytes make sense. Maybe it will, maybe not. – GioMac Aug 31 '13 at 0:21
  • Anyway, I'm marking this as the answer. It is more general and a slightly better solution if it is possible to do (I could have used it, my version is pretty much a shortcut version of this). – Simon Lindgren Aug 31 '13 at 0:24
  • aargh! :) your solution CAN break FS. Mine CANNOT :D "it should have been overwritten already", but you won't know how much of these 128K are overwritten by FS and if there is any file in these 4K blocks of FS. Erasing mdadm superblock can clear part of FS data. Up to 128K – GioMac Aug 31 '13 at 0:27
  • 1
    Yes. That's not what I was talking about. My hypothesis (which I'm not going to test, btw ;)) was that MD has a checksum of the data in the superblock. A partial overwrite of the superblock would change this checksum and make the superblock invalid. According to the mdadm man page, only valid superblocks are zeroed if --zero-superblock is run without -f. Therefore, the partial overwrite case shouldn't result in data loss in most cases. Note that I'm not debating which solution is better here (yours is). (Also again, I'm pretty new to md internals... I could be wrong about the checksum). – Simon Lindgren Aug 31 '13 at 0:35
2

This is how I figured this out (it might be quite specific to my case but I'll try to keep it general where I can).

(When I talk about devices, what I mean are the devices the raid volume is composed of, not the raid array itself)

I used mdadm -E $DEVICE to figure out which metadata format the array was using. I then went to [0] to find some information about the superblock format. In my case this was version 0.90.

This format has the superblock stored towards the end of the device. This is where my situation comes in. My old array was made directly on the drives, no partitioning. Because of this, I knew the superblock should be at the very end of the device. My new partitioning included a swap partition at the end. Therefore, there was not much data to lose where the superblock was located.

I did some reading around, the conclusion I reached was that mdadm --zero-superblock only zeroes out the superblock itself and thus it should be safe in my case. I went ahead and removed the superblocks on all three devices:

mdadm --stop $ONE_OF_THE_DEVICES

Repeat this line as required

mdadm --zero-superblock $DEVICE

Some additional comments/speculation:

Generally, if the space is needed by the new partitioning/filesystems it should have been overwritten already. Thus, if the superblock still there, zeroing it shouldn't hurt the partitioning/filesystems. I am however not sure how MD handles the case where the superblock has already been overwritten on one or many of the devices but not all. The man page says that -f is needed to zero the superblock out if it is invalid, but keep it in mind.

0: https://raid.wiki.kernel.org/index.php/RAID_superblock_formats

  • nope, it's not always safe, see my answer – GioMac Aug 30 '13 at 23:37
  • This was my description of how I worked out that it should be safe for my case. IE, there's nothing important there anyway, I knew where on the disk the superblock was, etc. – Simon Lindgren Aug 31 '13 at 0:14
  • it might break filesystem consistency. – GioMac Aug 31 '13 at 0:17
2

wipefs --all /dev/sd[4ppropr14t3][123] (of course set up the glob for your drives/partitions!)

  • 1
    Worked for me all tough maybe also related with a previous: "mdadm --manage --stop /dev/md5" instead of a simple "mdadm --stop /dev/md5" – luison Nov 28 '18 at 0:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.