2

We've been attempting to debug load programs on our servers, and along the way we've been looking at some graphs, like this one:

Memory graph

(The white spaces are when the server got so hosed we couldn't collect Munin stats.) I was curious to know why committed memory persistently drops down at approximately 4AM every night. Is there something obvious I'm missing? It's not like we reboot all of our user applications during that time...

Extra. This server is a shared web host. We handle about two thousand active users, and offer them CGI, FastCGI, SSH in any language they could possibly want. This server is running Fedora 13, and part of an active cluster of five servers. We use the networked filesystem AFS. We support cron but no user cronjobs run from this server (we give those to a different server.)

3
  • Are there any services being started or reloaded at that point in time?
    – Zoredache
    Oct 21, 2011 at 18:54
  • Not obviously. Our networked filesystem runs backups at 2AM, we rotate logs at around 3AM, if it's a user-service, they would have all needed to have decided to reload their applications simultaneously, which is pretty implausible. Oct 21, 2011 at 18:58
  • Correction: our AFS backups are staggered over many hours. They also have no meaningful impact on client performance. Oct 21, 2011 at 19:05

1 Answer 1

4

logrotate calls service httpd graceful, which kills any running FastCGI processes. Thanks for playing.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .