It's well-known that you should never fsck a mounted partition. I can understand how this could easily lead to corruption if the filesystem is written to by fsck (e.g., the -a option is used), but why can't read-only checks be run on mounted disks?
"Note that in general it is not safe to run
e2fsck on mounted filesystems. The only exception is if the
-n option is specified, and
-L options are not specified. However, even if it is safe to do so, the results printed by
e2fsck are not valid if the filesystem is mounted. If
e2fsck asks whether or not you should check a filesystem which is mounted, the only correct answer is ''no''. Only experts who really know what they are doing should consider answering this question in any other way. "
The basic problem is that the file system checker is (usually) not part of the file system. Instead it is a separate program that reads and writes to the same disk as the file system code in the kernel. As a result, if you run fsck on an active file system, you have two different entities that are reading (and potentially modifying) the same data (the disk), but they aren't coordinating with each other in any way. The result, as others have pointed out, is that most checkers expect that nobody else is changing the file system metadata while they run. They will get confused and/or report spurious errors if the kernel file system changes something that the checker doesn't expect.
There are a few file systems with checkers that are explicitly designed to be run "on-line" (i.e., while the file system is active). Newer versions of FFS/UFS do this by running fsck against a recent snapshot of the filesystem (a read-only, point-in-time, copy-on-write replica). If it finds problems, such as inconsistencies in the allocation bit-maps, it corrects them via system call, rather than by writing to the raw disk. This lets it coordinate with the active file system.
NetApp's WAFL also has an on-line checking tool. There are probably others.
Running fsck on a partition mounted read-write would be silly, even with fsck in read-only mode. The filesystem will change under fsck, and in-memory data that fsck caches from the filesystem will become invalid (and thus fsck will see inconsistency). You can run fsck on a read-only mounted filesystem in read-only mode and get valid results. Running fsck in read/write mode on a read-only mounted filesystem, if fsck makes changes to the filesystem during the course of its run, would cause the kernel to see filesystem structures changing unexpectedly underneath it. That would also be bad.
Apart from the fact that it would probably kill your I/O throughput, if the filesystem is being modified while it's being fsck'd then there's no way fsck could keep track of the changes and report inconstancies.
Some filesystems like XFS let you perform the check for consistency while the filesystem is mounted read-write, with the caveat that spurious errors will likely be reported.
xfs_check recommends the filesystem be unmounted or mounted read-only before performing the check.
Well, the point of fsck is to report filesystem inconsistencies, that is violated invariants.
However many of these checks involve more than one FS structure. If someone is modifying the FS (writing data), these structures may be temporarily out of sync. fsck would see this as an inconsistency, even though it is not really a problem. fsck has no way to tell if an inconsistency is just temporary, or a permanent problem that needs fixing. So this cannot possibly work (Unless a FS is designed specifically to allow online checking. Some do, but ext3 does not).
You can, just as you can stick your hand into a moving blender and possibly not injure yourself, or just as you can leap from a tall building while aiming for the small pile of cushions you laid out on the sidewalk below.
But why would you, other than to test your own mortality? Because your boss is certainly going to test it again when he finds out WHY the mail server won't recognize the root drive now.