Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We've recently setup "New Relic" on our web server. This is a great tool to measure full-stack performance.

The problem is we cannot get to run New relic for more than 10-15 minutes during our peak times. For some reason our website stops responding. And we have to stop the new relic daemon to bring the web server up (Apache).

An important log file for New Relic is "newrelic-daemon.log". This log shows the following line just before things come to a halt:

"2012-02-09 22:01:29 [26035/timer] error: RPM command 'metric_data' failed(6): Couldn't resolve host 'collector-7.newrelic.com'"

So I'm thinking the problem is with our own server. Where should I look to further investigate the problem?

UPDATE:
/etc/resolv.conf content:

search ourdomain.com
nameserver 94.101.94.4
nameserver 94.101.95.4
share|improve this question
    
That's a DNS error. Can you update your post with the contents of /etc/resolv.conf? –  cjc Feb 11 '12 at 12:13
    
How about if I add google "nameserver 8.8.8.8" to resolv.conf? Does this make sense? –  Haluk Feb 11 '12 at 12:40
1  
It turns out what stopped new relic was our httpd.conf. ServerLimit and MaxClients were both set to: 1024 . We reduced bothm of them to 950 and now all is working fine. –  Haluk Feb 14 '12 at 16:03
add comment

1 Answer

up vote 1 down vote accepted

Yes, try using "nameserver 8.8.8.8" and "nameserver 8.8.4.4". There might some other issues, possibly related to load, but that's a reasonable first try.

The IPs that you currently have are for public1.radore.net and public2.radore.net. Are those your hosting provider?

share|improve this answer
    
Yes those belong to the hosting provider. Still we received the same error. Maybe New Relic is reporting a wrong error. Having tried the google name servers, I'm not entirely sure anymore if the problem is with the nameservers. –  Haluk Feb 11 '12 at 14:19
    
What's the load on the boxes when NewRelic and Apache are non-responsive? I think it's less likely that DNS resolution is the issue. It's more likely it's the symptom of some other problem. –  cjc Feb 11 '12 at 14:21
    
Great question! That's exactly what I pin-pointed a few moments ago. New Relic process is consuming 100% CPU when Apache is unresponsive. Probably that's why Apache is unresponsive (and probably any other processes on the server for that matter.) Killing the New Relic process resolves the issue instantly. Normally New Relic does not consume 100% cpu. –  Haluk Feb 11 '12 at 14:23
    
OK, now you have to figure out why NewRelic is pegged. That might be a support email to them, actually. I've found them responsive in the past. –  cjc Feb 11 '12 at 14:32
    
According to the daemon logs 2012-02-13 15:43:12 [25947/listener] verbosedebug: accept() failed with: 24: Too many open files [repeated 8697 times] there seems to be an error related to the count of open files. –  Haluk Feb 13 '12 at 19:29
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.