Note that inode_cache & ext3_inode_cache slabs are very small compared to dentry_cache.
What happens is that slowly and steadily the within a week dentry_cache grows from 1M to ~5-6G
Then I need to run
echo 2 > /proc/sys/vm/drop_caches && echo 0 > /proc/sys/vm/drop_caches
This started to happening one day on all servers hosting some web code - the developers are saying that they have not changed anything related to filesystem access pattern around the time then the problem started.
The system is centos5 with 2.6.18 kernel so I don't have any instrumentation features available th newer kernels. Any I idea how I can debug the problem? maybe with systemtap? This is a ec2 instance - so not even sure that systemtap will work there.
Thanks Alex
[root@www1a ~]# cat /etc/fstab /dev/sda1 / ext3 defaults,noatime 0 0 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 [root@www1a ~]#