Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Last night at about 4:30am, both our app servers decided to have a seizure. We can't figure out why. Visitor load was low (100 live on the site) and there are no cron jobs scheduled on the servers.

We run ubuntu with apache, passenger and rails 3.0.3. The apache error log had a "cannot allocate memory" error right when the blue line dropped to the floor. Not sure what this tells me though since, the server runs well 99% of the time.

Any ideas for figuring out the root cause?

Thanks.

CPU Graph over the last 12 hours for our app servers

share|improve this question

2 Answers 2

Some robot indexing your pages? google?

Or an attacking script trying logins or searching for not secured apps or not updated services with known vulnerabilities?

share|improve this answer
    
Yes, time to hit the logs. The servers were pushed to 60% load for over an hour: there must be footprints left on your system somewhere. –  DutchUncle Feb 18 '11 at 19:21

Notice that you're using EC2. Are both app servers in the same EC2 availability zone? Were there any kernel messages relating to errors (via dmesg)?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.