0

So the latest configuration of our Nova compute nodes are using raw /dev/sdX devices (no labels nor partitions) as components for an md0 (raid0) array on which they're hosting an XFS filesystem. When one of the underlying hard disks fail, then the raid remains blissfully unaware of this.

This is confirmed by other cases like mdadm did not notice a failed disk in raid0

The question then arises. After we replace a failed hard disk, how do we reassemble this array without being forced to perform a new mkfs? Or would it be sufficient to fsck the filesystem and have it rediscover the (no-longer "bad" blocks)? Is that even a thing? (If the OS tries to use the blocks on the failed device I presume that the drivers have to simply return "bad blocks" for that entire range. Traditionally in Unix filesystems backblocks are forever ... you never try to reclaim them. Is there a switch to xfs_repair to force it to re-evaluate bad blocks?

Am I misunderstanding the underlying mechanics here?

3
  • do you mean raid1...because raid0 = no redundancy. Feb 10, 2016 at 3:22
  • I'm well aware that raid0 is no redundancy. In this case any VMs whose data was hosted on blocks affected by the drive failure have already died and been re-spawned by the customers. So the question concerns how to restore the FS capacity without impacting the VMs which were hosted on other portions of the FS. --- think "linear mode" RAID0 rather than interleaved.
    – Jim Dennis
    Feb 10, 2016 at 3:44
  • (The alternative, if there's no way to forced XFS to re-evaluate the "bad blocks" is to forced all my OpenStack customers to "drain" their VMs off that host so I can do a new mkfs on the md0 device).
    – Jim Dennis
    Feb 10, 2016 at 3:46

1 Answer 1

1

You can not. As you were said before, raid0 provides no redundancy, regardless of the fact is it interleaved or sequential. One disk in raid0 still functioning while the same is not it's basically the same thing when you have wiped out a [second] half of the non-raid0 disk: you still can read and probably write some of the sectors because they still contain formatting and valid data, but as soon as you want to do something with others the OS will fail.

So, if you are insisting to continue to use the non-documented backsides of the raid0 failure, presenting them as design benefits, you have two choices: write some additional software [layers] yourself or meet your doom, because there's no methods ready to use.

5
  • The OS (Linux) won't "fail" the block device driver will return I/O errors. The filesystem driver will (presumably) mark those blocks as "bad" and the OS will return I/O errors to any user space processes (or kernel threads?) which have made system calls attempting to read data from the filesystem blocks serviced by that (physical device). As I've said, I know that any VMs or other processes which data on the affected portions of the OS will die (or at least have to handle their own I/O errors). I'm asking if there's a way to repair to FS so that the "bad" regions are reclaimed.
    – Jim Dennis
    Feb 10, 2016 at 21:24
  • You are deeply inside a logical trap. There's no so-called bad blocks. One of your disks is missing the formatting markup entirely. Like dead. The fact that you replaced the real dead disk with an alive one doesn't change anything. There would be a way for non-interleaved non-redundant arrays if xfs was shrinkable, but it's not.
    – drookie
    Feb 10, 2016 at 21:41
  • md0 is a raid0 of sda sdb ... sdx; there is a filesystem (in this case XFS) on md0. one of the disks (sda) dies. The rest of the filesystem is still there, alive, active, the OS continues to allow access to it; most of the processes accessing data on the filesystem are unaffected. Only those which access portions of the filesystem that reside on sda see I/O errors (unless a critical portion of the FS is affected and causes a kernel panic or read-only remount --- which I'm not seeing in my real-work exemplar ... a system which is in this state even as I'm typing this).
    – Jim Dennis
    Feb 11, 2016 at 0:54
  • I have posited (but not confirmed) that the a side effect of this situation (any attempts to write to the ports of the filesystem on md0 which exist on sda) is for the FS driver to treat those as "bad blocks." This is the essence of my question. Is it possible to "repair" the filesystem such that the data across the other X-1 disks in this array does not have to be moved around?
    – Jim Dennis
    Feb 11, 2016 at 0:57
  • Yeah, never surrender, stick in declining.
    – drookie
    Feb 11, 2016 at 6:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .