What are your checklist/routine when setting up a Linux web server?
What do you recommend to achieve maximum security?
Is there any preferred way to perform repeated maintenance?
Server Fault is a question and answer site for system and network administrators. It only takes a minute to sign up.Sign up to join this community
First of all, be aware that any scripting ability in Apache (php, cgi, ruby,...) is the potential equivalent of a shell account with privileges of the user running the script.
Virtualize or chroot apache, so that any compromise is at least somewhat contained in an additional layer of security. Be aware that when you chroot apache, maintenance may become harder, as you end up moving libraries to the jail etc. If you're on FreeBSD you can use a jail instead, which is much easier to maintain, since you can just install apache from ports, and run portaudit from within it, without having to worry about any library dependencies and moving files manually, which always becomes an ugly mess. With BSD jails you can simply keep using the package management system (ports). (On GNU/Linux you can also use VServer for virtualization. - Suggested by David Schmitt )
(obviously) Keep up with updates and patches, not only for Apache, but also PHP, ruby, perl, etc... don't just trust your OS to give you all the updates either. Some distro's are extremely slow with their patches. Limit exposure time to 0-day vulnerabilities as much as possible. Stick the milw0rm feed in your RSS reader, subscribe to the insecure.org mailing lists, etc... Not only will it help you learn about vulnerabilities before your OS gets around to releasing a patch, you will also learn about vulnerabilities in certain php cms applications for example, which may not even be managed or patched by your OS at all.
Using stuff like suhosin to protect php helps. But also learn to understand it, tweak it's config to your application's expected parameters.
Using a kernel patch such as PaX may help protect you from many buffer overflow vulnerabilities. Even if your software is vulnerable. (This does not make you invulnerable, it's just yet another, minor, layer.)
Don't get over-confident when using some security tool. Understand the tools you use, and use common sense. Read, learn, keep up with as much as you can.
Consider using mandatory access control (eg: SELinux). It allows you to specify, for each application, what it is allowed to do, in great detail. What files is it allowed to access. What kernel calls is it allowed to make, etc. This is a very involved process and requires lots of understanding. Some distro's provide pre-made SELinux policies for their packages (eg: Gentoo ). This suggestion is kind of a contradiction to the one below, but still valid, nevertheless.
Keep things simple. A complex security strategy may work against you.
In Apache, set up a very restrictive default rules (Options None, Deny from all, etc...) and override as needed for specific VirtualHosts.
Deny access to all dotfiles (which also immediately covers .htaccess files)
Always use https anywhere there is any sort of password authentication.
Firewall should be a deny-by-default policy. Build some specific rules in your firewall to log specific traffic.
Set up log parsing scripts to scan your logs for anomalies. (the prelude IDS suite can do this, but honestly, I recommend you build up your own scripts over time, as it will help you understand your own tools and rules better.)
Have the server mail you daily reports on last logged in users, active connections, bandwidth used, etc...
Have a cron scan for suid binaries, world writeable files, and stuff like that, and have them mailed to you.
For any of the stuff you set up that gets mailed to you, you should build up a list of exceptions over time. (folders to ignore filesystem changes on, 777 files to allow, suid binaries to allow). It is important that you only get notified of things that shouldn't happen. If you get a mail every day with trivial stuff, you will start to ignore them, and they will become pointless.
Have a good solid layered redundant backup strategy. And don't just assume that making an image or copy of everything works. For example, if MySQL is in the middle of writing to a table during your backup, your MySQL binary files may be corrupted when you restore your backup. So you will need a cron that mysqldump's your databases on top of regular images or nightly tarballs or version control or whatever else you have setup. Think about your backup strategy. I mean, REALLY think about it.
Don't rely on lists like this for security :) Seriously! You'll find lots of these all over the internet, go read them all, research every suggestion, and use common sense and experience to make up your own mind. In the end, experience and common sense are the only things that will save you. Not lists, nor tools. Do read, but don't just copy without understanding.
I recommend the Linux Security Checklist, from SAN. I use that, plus another in-house procedures. The checklist might be a bit out-dated, but many of the key points stand true.
There will always be countless permissions to check, countless checklists, never ending discovery of new bugs/vulnerabilities. Security I wouldn't think is something you turn on or off, it is something you do constantly.
Given the "inevitable failure" of software, SELinux helps put out some worries (again no silver bullet for security). assuming a userspace application is compromised the correct SELinux policy will prevent it acting at its usual (ie if SELinux was disabled or permissive) privileges. This of course will require you to monitor your audit logs and analyze installed policy and modify it where necessary to allow applications to function.
Not saying default policy wouldn't help but I personally like to know what it allows.