So on my linux server which runs Ubuntu 12.04, I have a folder in there that has ~3000 folders and within those folders they contain about 3000 text files each. In total, this takes roughly 80 GB.
So whenever I ls into any of these folders, the systems starts to get horribly slow. The system is a Xeon E5-2620 w/ 64 gb of RAM and the hard drive is on is a basic 7200 RPM Seagate HDD (don't remember if it's SATA 3.0 or not).
I'm also currently deleting the files, because they need to be regenerated. The rm process is taking fairly slow, however I'm just running a multithreaded rm -rf so there could probably be some improvement there.
Any tips you guys could give to speed this all up a bit? Thanks