Frequent dirty file systems

Question

Good morning,

I'm seeing an issue on a few servers that they are going 'dirty' a lot, around once a month.

For example: This was on 11/8:

# tune2fs -l /dev/sda7 | grep state
Filesystem state:         clean with errors

We ran a fsck on home, and it cleaned up fine:

12552126 inodes used (11.38%)
  628394 non-contiguous inodes (1.6%)
         # of inodes with ind/dind/tind blocks: 589193/11935/1
109892528 blocks used (49.83%)
       0 bad blocks
       4 large files

11673859 regular files
  867367 directories
       0 character device files
       0 block device files
       0 fifos
    1242 links
   10891 symbolic links (10883 fast symbolic links)
       0 sockets
--------
12553359 files

However, now it's dirty again!:

# tune2fs -l /dev/sda7 | grep state
Filesystem state:         clean with errors

# grep EXT3 /var/log/messages
Dec 16 20:22:44 server kernel: EXT3-fs error (device sda7): ext3_lookup: deleted inode referenced: 105628446
Dec 16 20:22:44 server kernel: EXT3-fs error (device sda7): ext3_lookup: deleted inode referenced: 105628446

I'm honestly not sure where to go from here. Any suggestions / pointers are greatly appreciated.

Happy holidays!

Answer 1

Have you also looked for i/o errors in /var/log/messages, /var/log/syslog, and /var/log/dmesg?

How busy are these filesystems? If you get a lot of reads & writes and file creation/deletion (particularly below the filesystem's block size) you can get this sort of behaviour.