up vote 0 down vote favorite
1
share [g+] share [fb]

After 1 or 2 days, my firewall becomes extremely slow. I have 5 machines behind it and they also become extremely slow. After hardware reboot, it becomes fast again.

My current conntrack limit:

[root@fw ~]# cat /proc/sys/net/ipv4/ip_conntrack_max
100000

Average conntrack table count:

[root@fw ~]# cat /proc/net/ip_conntrack |wc -l
1301

My iptables rules:

    [root@fw ~]# iptables -L -n
    Chain INPUT (policy ACCEPT)
    target     prot opt source               destination

    Chain FORWARD (policy DROP)
    target     prot opt source               destination
    DROP       all  --  24.0.0.0/8           0.0.0.0/0
    ACCEPT     udp  --  $IP_155        0.0.0.0/0           udp
    ACCEPT     udp  --  0.0.0.0/0            $IP_155       udp dpt:8888
    ACCEPT     udp  --  0.0.0.0/0            $IP_155       udp dpt:9999
    ACCEPT     udp  --  0.0.0.0/0            $IP_155       udp dpt:8777
    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0
    ACCEPT     all  --  $IP_160        $IP_39
    ACCEPT     all  --  $IP_39         $IP_160
    ACCEPT     all  --  $ip_veloX        0.0.0.0/0
    ACCEPT     all  --  192.168.1.0/24       0.0.0.0/0
    ACCEPT     all  --  192.168.1.0/24       0.0.0.0/0
    ACCEPT     all  --  0.0.0.0/0            192.168.1.0/24
    ACCEPT     all  --  192.168.1.0/24       0.0.0.0/0
    ACCEPT     all  --  0.0.0.0/0            192.168.1.0/24
    ACCEPT     all  --  192.168.1.0/24       0.0.0.0/0
    ACCEPT     all  --  0.0.0.0/0            192.168.1.0/24
    ACCEPT     tcp  --  0.0.0.0/0            $IP_39        state NEW tcp dpt:2106
    ACCEPT     tcp  --  0.0.0.0/0            $IP_160       state NEW tcp dpt:2106
    ACCEPT     tcp  --  0.0.0.0/0            $IP_156       state NEW multiport dports 7777
    ACCEPT     tcp  --  0.0.0.0/0            $IP_211       state NEW multiport dports 7777
    ACCEPT     tcp  --  0.0.0.0/0            $IP_215       state NEW multiport dports 7777
    ACCEPT     tcp  --  0.0.0.0/0            $IP_160       state NEW multiport dports 7777
    ACCEPT     tcp  --  0.0.0.0/0            $IP_155       state NEW multiport dports 7777
    ACCEPT     tcp  --  0.0.0.0/0            $IP_155       state NEW multiport dports 8767,9999,8777,14534,51234,6969,7777
    ACCEPT     tcp  --  0.0.0.0/0            $IP_160       state RELATED,ESTABLISHED
    ACCEPT     tcp  --  0.0.0.0/0            $IP_156       state RELATED,ESTABLISHED
    ACCEPT     tcp  --  0.0.0.0/0            $IP_211       state RELATED,ESTABLISHED
    ACCEPT     tcp  --  0.0.0.0/0            $IP_215       state RELATED,ESTABLISHED
    ACCEPT     tcp  --  0.0.0.0/0            $IP_39        state RELATED,ESTABLISHED
    ACCEPT     tcp  --  0.0.0.0/0            $IP_155       state RELATED,ESTABLISHED
    ACCEPT     tcp  --  $IP_160        0.0.0.0/0           state RELATED,ESTABLISHED
    ACCEPT     tcp  --  $IP_156        0.0.0.0/0           state RELATED,ESTABLISHED
    ACCEPT     tcp  --  $IP_211        0.0.0.0/0           state RELATED,ESTABLISHED
    ACCEPT     tcp  --  $IP_215        0.0.0.0/0           state RELATED,ESTABLISHED
    ACCEPT     tcp  --  $IP_39         0.0.0.0/0           state RELATED,ESTABLISHED
    ACCEPT     tcp  --  $IP_155        0.0.0.0/0           state RELATED,ESTABLISHED
    ACCEPT     tcp  --  $IP_160        0.0.0.0/0           state NEW
    ACCEPT     tcp  --  $IP_156        0.0.0.0/0           state NEW
    ACCEPT     tcp  --  $IP_211        0.0.0.0/0           state NEW
    ACCEPT     tcp  --  $IP_215        0.0.0.0/0           state NEW
    ACCEPT     tcp  --  $IP_39         0.0.0.0/0           state NEW
    ACCEPT     tcp  --  $IP_155        0.0.0.0/0           state NEW

Chain OUTPUT (policy ACCEPT)

    target     prot opt source               destination
    [root@fw ~]#

[root@fw ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 142G 21G 114G 16% /

link|improve this question
How is your free memory? – Matt Simmons Jun 5 '09 at 12:27
Also what is your free disk space? You aren't filling up because of some log file, are you? – Andy May Jun 5 '09 at 12:34
I have enough disk space.. [root@fw ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 142G 21G 114G 16% / tmpfs 501M 0 501M 0% /dev/shm I have 1GB RAM.. I believe it can indeed be a RAM issue.. I will check it when lag arrives.. Thanks!! – Filipe Jun 5 '09 at 12:41
and yes - what exactly slow menas? irresponsive when used from console? high latency on local lan segment? – pQd Jun 5 '09 at 13:46
feedback

3 Answers

up vote 3 down vote

it'll not help directly in this case but always put 

iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT

as first rule. this will save you some cpu cycles for all packets belonging to connections already established.

similarly with input / output if you have DROP policy there.

what is load of the machine [ take it from w or top ]

do you have something interesting in dmesg?

do you do some address translation?

link|improve this answer
Thanks for the tip! Load is always under ~ 0.3 dmesg seems to be ok.. The only other rule i forgot is this: [root@fw ~]# iptables -t mangle -L Chain PREROUTING (policy ACCEPT) target prot opt source destination TOS tcp -- anywhere anywhere tcp dpt:cbt TOS set Minimize-Delay I think it is up to RAM.. I am increasing it and hopefuly it will not happen again.. thank you all for the help.. first time i use this site and all i can say is this site is amazing – Filipe Jun 5 '09 at 13:06
1  
@Filipe : install munin munin.projects.linpro.no it's amazing how many things become apparent when you take a look at 'big picture' with weekly/monthly trends. – pQd Jun 5 '09 at 13:24
feedback
up vote 2 down vote

Provided that you're referring to your bandwidth becoming slow, I would first try reloading iptables (root$ /etc/init.d/iptables reload, or alternatively flushing and reloading your rules manually) and see if that solves your problem. If it does, then you know it's your fw config. If it doesn't then you can start monitoring your system to see if something is eating up resources.

I'm guessing the former problem, and it's generally related to your nat translation table getting full. You can view it with '-t nat' and whatever appropriate options.

link|improve this answer
+1 for solid troubleshooting! Check individual services to isolate the problem. Are there other services he might try? And if iptables is the culprit is there a setting change to limit the log size or periodically flush it? – tomjedrz Jun 5 '09 at 13:43
feedback
up vote 2 down vote

Along with the other suggested possibilities, another one is that one of your gateways NICs is starting to fail. When things start to "slow down", run and ifconfig and check the packet counts. In particular check the error counts.

link|improve this answer
or try re-initializing the interfaces (down them, then up them) – derobert Jun 5 '09 at 18:08
feedback

Your Answer

 
or
required, but never shown
discard