Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

How many logs/second can usually be handled by syslog servers? By syslog servers, I am referring to rsyslog, syslog-ng, splunk etc. The intent of the question is to find out at what logs/second rate the OS (Linux kernel >=3.0) becomes bottle-neck.

So far, I am able to forward about 10,000 logs/sec using UDP. If I increase the rate, logs start getting dropped. I don't know what is causing the drops, is it the application (it is a custom build log server) or is it the OS?

Can you please share your experience with regards to how many logs/second you are able to transfer without seeing any drops?

share|improve this question
add comment

2 Answers

up vote 0 down vote accepted

You can increase the kernel UDP buffers, but this will not solve the problem. If you want to forward this amount of messages reliably, you should forget UDP syslog and use TCP instead. We have measured over 500k events per second with nxlog when transferring syslog messages over TCP.

share|improve this answer
add comment

This question can not be answered simply in its current form. There is no general rule for the upper limit of logs transfer rate. This can be varied based on several factors like your network configuration/reliability (between sender and recipient), machines specs, and whether they are loaded or not, etc...

UDP is not reliable protocol. You can try using TCP as it is connection-oriented protocol can be used for reliable data transmission. It imposes a little overhead more than UDP, but it may worth trying.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.