we have a syslog-ng server acting as an intermediary logger.
the server receives data from all devices on the network (from many different sources, on both UDP & TCP ports), filters them a bit, and forwards the data to a SIEM (splunk).
when looking at ifconfig stats, i can see this :
[root@xxxxxxxx ~]# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:50:56:00:00:00 inet addr:126.96.36.199 Bcast:188.8.131.52 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:60451021996 errors:0 dropped:0 overruns:0 frame:0 TX packets:158501574 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:5000 RX bytes:12508573129969 (11.3 TiB) TX bytes:220148785267 (205.0 GiB)
this part is quite shocking to me :
RX bytes:12508573129969 (11.3 TiB) TX bytes:220148785267 (205.0 GiB)
obviously, the server received around 11 Terrabytes of data since the last boot (15 days uptime) but "only" transferred a little part of it ? (250 Gbs).
is there a way for me to check how is this possible ? i know there's a bunch of UDP dropped messages, but is it taken in consideration ? (dropped is displayed 0).
i expect a little less TX (due to the fact that we filter some messages), but not that much (10% max).
For information, Netstat on UDP shows :
Udp: 10903564401 packets received 8401685 packets to unknown port received. 49356622070 packet receive errors 43665773 packets sent RcvbufErrors: 34287641
any tips on how to investigate this ? thanks !
iptraf, which also tells you how much traffic comes from and goes to which IP addresses.
netstat -s? If not and syslog is keeping up, then maybe this is just sysctl socket buffer tuning. Are you forwarding to splunk with udp or tcp? Syslog-ng itself also has buffer settings and if using tcp, there could be rate limits on the splunk forwarder / listener.