Best Linux network tuning tips?

Question

I figure it's a good idea to have a central question for linux network tuning. I'm particularly interested in tuning options for the following.

1. Web server
2. Varnish / Squid server
3. DB server
4. File server
5. App server

Answer 1

At this point it's generally not worth the overhead of "tuning" within the kernel. Unless you have the test gear (or a cluster to A/B options on) and very good documentation it's not worth it for the sys-admin overhead alone.

These days the linux kernel has good auto-tuning of TCP, you can do better, however those are not the default because they're seen to be "greedy" and not fair on the Internet.

As close as I get to tuning is:

1. Mount noatime (implies nodiratime), using ext3 or xfs
2. Run only needed services
3. For services like squid & mysql allow them to use more RAM

With rare exceptions those three will get you 90% there.

Answer 2

network stack tuning i put on routers/firewalls connecting to net hundreds of users

net/ipv4/route/max_size=262144
net/ipv4/netfilter/ip_conntrack_tcp_timeout_established=18000
net/ipv4/neigh/default/gc_thresh1=1024
net/ipv4/neigh/default/gc_thresh2=2048
net/ipv4/neigh/default/gc_thresh3=4096
net/netfilter/nf_conntrack_max=128000
net/netfilter/nf_conntrack_expect_max=4096

[ mostly empirically tweaked over years based on some unpleasant crashes ]

for file/sql/poroxy server you definitivly want to mount partition with data/spool with noatime,nodiratime and for reiserfs - notail ] to avoid additional disk trashing whenever file is accessed for read.

and you really want [ maybe you don't know about it yet ;-] gather statistics - eg with munin - to do some capacity planning and see if your tuning indeed decreases system load.

Answer 3

One thing you may want to look at web100. It is a kernel patch, and a userland too designed to optimized the Linux kernel for high-performance networks.

The Web100 project is aimed at producing a complete host-software environment that will run common Web applications at 100% of the available bandwidth, regardless of the magnitude of a network's capability.

Also check out High Performance SSH/SCP.

SCP and the underlying SSH2 protocol implementation in OpenSSH is network performance limited by statically defined internal flow control buffers. These buffers often end up acting as a bottleneck for network throughput of SCP, especially on long and high bandwith network links. Modifying the ssh code to allow the buffers to be defined at run time eliminates this bottleneck.