23

I tried to mount a formerly readonly mounted filesystem read-writeable:

mount -o remount,rw /mountpoint

Unfortunately it did not work:

mount: /mountpoint not mounted already, or bad option

dmesg reports:

[2570543.520449] EXT4-fs (dm-0): Couldn't remount RDWR because of unprocessed orphan inode list.  Please umount/remount instead

A umount does not work, too:

umount /mountpoint
umount: /mountpoint: device is busy.
    (In some cases useful info about processes that use
     the device is found by lsof(8) or fuser(1))

Unfortunately neither lsof of fuser don't show any process accessing something located under the mount point.

So - how can I clean up this unprocessed orphan list to be able to mount the filesystem again without rebooting the computer?

4
  • 1
    Have you tried fuser -km /mountpoint yet? Beware though, the -k flag will kill all processes accessing that directory. May 31, 2012 at 0:32
  • Can you provide a little bit more insight to what dm-0 consists of?
    – thinice
    May 31, 2012 at 1:28
  • I have feeling I know whats up, but can you tell me, was the filesystem originally rw, remounted (due to ata error or whatever) ro, and now you are trying to rw again? Jul 1, 2012 at 22:47
  • @Mlfe: The filesystem was formerly remountend ro by purpose. It's a filesystem on an LVM holding a daily backup snapshot that will be set to rw during backup operation and ro after finishing the backup.
    – bmk
    Jul 9, 2012 at 7:39

6 Answers 6

6

You clean up the unprocessed orphan inode list by unmounting and remounting the filesystem.

An extended discussion from the linux-ext4 mailing list has more information about what this message is and why it may appear. In short, one of two things has happened: Either you've run into a kernel bug, or much more likely, some filesystem corruption happened one of the previous times you remounted the filesystem readonly. Which is probably why the system thinks something is still using the filesystem when there isn't.

If it's been a year and you still haven't rebooted the machine, just give up and schedule a maintenance window.

1
  • Meanwhile I scheduled a maintenance window and rebooted the machine. That solved the problem (I didn't expect anything else...). I will accept your answer. Probably you are right that there was some filesystem corruption - although I cannot prove that.
    – bmk
    Aug 28, 2012 at 16:38
35

If you're using ext2 / ext3 / ext4 you should be able to use e2fsck to clean up orphaned inodes:

e2fsck -f

For reiserfs, you can use reiserfsck which will also clean up orphaned inodes.

7
  • 2
    Thanks thanks a lot.. I spend hours figuring out the error. Doing 'e2fsck -f /dev/sda1' fixed the orphaned nodes for me along with some other fixes. I just said yes to all and works fine now :)
    – whitehat
    Jul 11, 2016 at 15:26
  • 1
    Thanks a lot!!. Yours commands fixed readonly VirtualBox VM disc after unsucessfull new VirtualBox version install: sudo e2fsck -f /dev/sda1
    – Andrew
    Aug 9, 2017 at 22:52
  • 2
    Perfect, worked for me on root partition. The accepted answer (reboot) did not work alone. I did have to reboot after e2fsck so seems like you do still need a maintenance window.
    – AdamS
    Sep 1, 2017 at 8:05
  • 2
    Better answer than the accepted one. That worked perfectly for my VPS. Found a lot errors and fixed it, than reboot and everything is running again. Saved my day. Nov 6, 2017 at 9:37
  • 1
    this worked for me. :yay:
    – deepdive
    Sep 9, 2019 at 5:39
10

e2fsck -f <mount point> won't work.

First find out the mount points with

sudo mount -l

Then fsck the drive directly.

For example for me

sudo e2fsck -f /dev/xvda2
1
  • 5
    When you google a problem and arrive at your own solution on stackoverflow. My life is now complete. Oct 3, 2019 at 4:01
1

I would recommend to first unmount the partition forcefully, i.e. using the -f option, and the running a file system check using fsck.

1
  • 1
    Unfortunately umount -f didn't succeed, too. The error message is the same as with a plain umount.
    – bmk
    Jun 8, 2011 at 11:55
1

You should probably try a lazy unmount, i.e:

umount -l
0

I was facing the same issue on an AWS EC2 machine. To complicate the resolution, the volume that was affected was the root volume of the EC2 instance. Hence the device was failing to boot and SSH was also not possible to the instance.

The following steps helped me resolve the issue:

  1. Detach the volume from the EC2 instance.
  2. Configure a new EC2 instance using the same AMI and in the same AZ as that of the old one.
  3. Attach the volume (detached in Step 1) to the new instance.
  4. Execute the following commands:
# Switch to Root user:
sudo -i

# Identify the device Filesystem name and save it as a variable:
lsblk
rescuedev=/dev/xvdf1    # Mention the right Filesystem for the particular volume.

# Use /mnt as the mount point:
rescuemnt=/mnt
mkdir -p $rescuemnt
mount $rescuedev $rescuemnt

# Mount special file systems and change the root directory (chroot) to the newly mounted file system:
for i in proc sys dev run; do mount --bind /$i $rescuemnt/$i ; done
chroot $rescuemnt

# Download, install and execute EC2Rescue tool for Linux to fix the issues: 
curl -O https://s3.amazonaws.com/ec2rescuelinux/ec2rl.tgz
tar -xf ec2rl.tgz
cd ec2rl-<version_number>
./ec2rl run
cat /var/tmp/ec2rl/*/Main.log | more
./ec2rl run --remediate

# Switch back from the Root user and unmount the volume:
exit
umount $rescuemnt/{proc,sys,dev,run,}
  1. Shut down the EC2 instance and detach the volume.
  2. Attach the volume to the original instance and start the EC2 instance.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .