Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

So our Linux admin left our project and suddenly I (the web coder with basic linux/server knowledge) am responsible for our dedicated server (Ubuntu Server) mainly running a web site (apache/mysql/php) and mail (Postfix). Our admin wasn't really a pro Linux admin, but rather some guy with basic Linux knowledge who figured out stuff as he went along. So I expect funky configurations, insecure services, etc.

My questions are:

  1. How do I perform an "audit" of the server to figure out it's current state, to make sure things are properly configured, that there are no unnecessary user accounts, that we aren't running unnecessary services, etc, etc.

  2. I'm unsure about how to back up our production web site. Besides the actual CMS files and db, there are apache configurations, mail databases and more that need back up. Any suggestion on how to automate this?

  3. What are the most important everyday duties of a linux admin that I absolutely have to perform? Huge question, I know.

share|improve this question

closed as not constructive by Zypher Jan 20 '12 at 17:54

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

    
What do you mean by a "funky" configuration? When my generation used the word it meant something was good. –  John Gardeniers Mar 7 '10 at 8:40
add comment

5 Answers

up vote 11 down vote accepted

Wow. Where to start.

This is what I would do but hopefully others will jump in with more/better suggestions.

First don't panic. I am assuming you are now root. You are currently the most dangerous threat to the server right now as you have a lot of power and not much idea of what to do with it.

Write down what services the server should be running. You know apache, mysql and postfix are required. I guess you may have an ftp server on there and you are able to ssh in so you need sshd running. Write down what services are installed. The quickest way to find out is probably by listing /etc/init.d/*. You then need to find out what are running. I don't know what the equivalent to the Red Hat chkconfig is, but failing an alternative ps -ef will list what processes are currently running. Also find out if a firewall is installed (such as iptables) and how it is configured.

Next get a list of all the cronjobs that are run. You probably don't need to worry too much about what they do right away but you should have a general idea of what the server is doing at different times.

Again I would write all this down.

Now write down who should have access to the server and who is allowed to have root access. Get a list of users who do have accounts from /etc/passwd.

Do a similar thing for FTP access, and other services if relevant like Subversion or remote MySQL connections.

Now you know a bit more about what your server is doing and who can access it, you should move on to how well it is doing. Check the log files in /var/log, especially /var/log/messages and spend some time looking for any errors.

Check if there are any outstanding updates to do using apt-get update && apt-get upgrade

When prompted to update choose no for now.

So far you should have made no changes.

You now need to review the information you have collected and decide what (if anything) needs fixing. The priorities are attempted cracking attempts in /var/log/auth.log, shutting down uneccessary services and tightening the firewall.

Make copies of all files before you edit them and test changes often so you can easily back out if something breaks.

Backups

You will need to decide what needs to be backed up. Obvious candidates are databases, /home/ /etc/ /var/log/ /var/spool/cron/ /var/www/ and any custom scripts called by crontabs. Then most people write a shell script to back this up locally and then use something like rsync to copy the files to another machine USB drive.

Day to day duties will include, checking the log files for any problems (check out logwatch to help you), performing security updates, checking backups and goinf forward setting up monitoring like MRTG and Nagios to eventually take the heavy lifting out of being an admin.

I would not worry too much though. It may seem daunting but that is because you are asking for it all in one go. The server is probably fine as it is, keep an eye on the logs and apply updates as they are released, plan what you want to do and work towards it, take little steps and try to enjoy it.

share|improve this answer
3  
Before doing anything else, I'd image and/or P2V that sucker. –  Scandalon Mar 5 '10 at 22:01
    
+1 for getting a current snapshot. Tarball the server over SSH or rsync at the very least. –  Gerald Combs Mar 5 '10 at 22:36
1  
When grabbing what services are installed, also run "dpkg -l > ~/installed_software". This will write a list the packages that were installed via apt-get to a file named "installed_software" in your home directory. Check for any services that are running that were not installed via apt-get. The old admin may not have installed the servers via apt-get, and those services will have to be updated through means other than "apt-get upgrade". One would think this would be rare, but it is not. –  Dru Mar 7 '10 at 6:35
    
Another suitable suggestion would be to get the configuration files under some form of version control. Then as you make adjustments or change things to suit your way of thinking you can track those changes and see exactly where you screwed up. Or maybe you're lucky, but it's still good advice. –  OldTroll Jun 3 '11 at 21:17
    
All of this is great advice. In addition to the above I'd run netstat -tanp and/or -anp to show details on active and listening connections and their associated processes. –  nedm Jun 3 '11 at 21:23
add comment

Some things to get you started:

  1. get somebody who has already done such an audit - or at least has some knowledge about the common pitfalls maintaining a server. Seriously - it pays off.

  2. Back up as good as you can and try to reconstruct a spare server - might be a virtual instance somewhere - until you're confident that a) you have backed up all important stuff and b) you're able to reconstruct the spare server from your backup in as little time as you desire. To add karma: exchange the current production server with your spare server. As long as you've not demonstrated that you can reconstruct from your backup, act as if you have none.

  3. Update, read security notices, keep an eye on the logfiles and automate the heck out of this once you know what to look for.

share|improve this answer
3  
+1 for finding someone who knows what they are doing. It will be cheaper in the long run. –  David Mackintosh Mar 6 '10 at 3:13
1  
+ 1 for the same but I'd go even further and suggest getting someone who is demonstrably more expert to make sure the whole system is in an appropriate state. A live system is no place to be learning pretty much from scratch. –  John Gardeniers Mar 7 '10 at 8:45
    
fully agreed, but as the system is already live, I guess there's no way around learning on a live system. –  Olaf Mar 7 '10 at 15:48
add comment

To more easily get a handle on the log files, you might consider installing (or activating..I dunno if ubuntu has this in the default install) LOGWATCH. It's very nice for giving you a summary sent to email every day. It usually picks up some funky stuff that a glance @ configs may not uncover.

share|improve this answer
add comment

If you value your server and it's data get help. Get someone to audit it.

If you don't know what "right" looks like then it can be hard to spot where something is "wrong" (or as you put it "funky"). Once someone the server is in a known good state.

Using something like VMWare Converter to make a staging VM of the server is a GREAT idea, you should look into that.

You then can poke around on the staging VM (copy of the server) and try to do each thing you are asked to do on the VM staging server first before the Production server.

share|improve this answer
add comment

After doing what Richard Holloway says; then do a network scan of the system to check what services are being provided by the server and check this against the data you have so far. It is possible to do really interesting things with linux that are hard to find just looking at logs.

I suggest using Zenmap from another system on the same network and getting whatever clearances you need from your bosses first. Zenmap is simple to install, is/has a GUI and doesn't try to exploit anything that is found.

share|improve this answer
add comment