we have a syslog-ng server acting as an intermediary logger.
the server receives data from all devices on the network (from many different sources, on both UDP & TCP ports), filters them a bit, and forwards the data to a SIEM (splunk).
when looking at ifconfig stats, i can see this :
[root@xxxxxxxx ~]# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:50:56:00:00:00
inet addr:1.1.1.1 Bcast:1.1.1.1 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:60451021996 errors:0 dropped:0 overruns:0 frame:0
TX packets:158501574 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:5000
RX bytes:12508573129969 (11.3 TiB) TX bytes:220148785267 (205.0 GiB)
this part is quite shocking to me :
RX bytes:12508573129969 (11.3 TiB) TX bytes:220148785267 (205.0 GiB)
obviously, the server received around 11 Terrabytes of data since the last boot (15 days uptime) but "only" transferred a little part of it ? (250 Gbs).
is there a way for me to check how is this possible ? i know there's a bunch of UDP dropped messages, but is it taken in consideration ? (dropped is displayed 0).
i expect a little less TX (due to the fact that we filter some messages), but not that much (10% max).
For information, Netstat on UDP shows :
Udp:
10903564401 packets received
8401685 packets to unknown port received.
49356622070 packet receive errors
43665773 packets sent
RcvbufErrors: 34287641
any tips on how to investigate this ? thanks !
iptraf
, which also tells you how much traffic comes from and goes to which IP addresses.netstat -s
? If not and syslog is keeping up, then maybe this is just sysctl socket buffer tuning. Are you forwarding to splunk with udp or tcp? Syslog-ng itself also has buffer settings and if using tcp, there could be rate limits on the splunk forwarder / listener.