Possible Duplicate:
My server's been hacked EMERGENCY

I have a small virtual server running Ubuntu set up for the personal use of myself and a friend. The server runs Postfix and Dovecot for Email, Apache, Tomcat and sshd for git and subversion access.

My friend gave a shell account to somebody else to enable them to access our Subversion repository. Unfortunately, it seems that this person was a bit lenient in their password choice. When I ran top on the server this week, I noticed a perl process camouflaged as /usr/sbin/httpd that was consuming 50% of CPU. lsof -i revealed that the process kept a connection open to some IP address in China. So my guess is that the user account was compromised by SSH brute-forcing and used to send spam or similiar activities.

As a first step, I disabled all ssh access for non-me users in the hope to prevent further abuse (root ssh access is disabled anyway). Now I am facing the question what to do to restore the server to use.

My plan this far looks like this:

  1. Dump the contents of the server to a tar file somewhere
  2. Reinstall the server from a clean image
  3. Take steps to prevent the server from being compromised again, such as disabling password-based ssh access entirely and using public keys instead
  4. Extract important data from the tar dump, make sure that it is clean and re-deploy it on the server and / or store it somewhere else

Is this a feasible approach? Have I missed any important steps? Something else to consider?

Some more specific questions:

  • It is possible that only the user account was compromised and the rest of the server is ok. In this case, a re-install is not necessary. Is there a safe way to tell if this is the case?
  • Do I have to notify my provider or authorities?
  • What is the best strategy to tackle step 4 - how can I tell malicious data from good data with regard to a linux system?

marked as duplicate by Bryan, EEAA, Sven, Shane Madden, Ward May 25 '12 at 7:00

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • Sounds about right. Don't copy any executable code from your tar and reinstall all web apps from a repository. Use the tarball as a reference for configs, etc. – uSlackr May 24 '12 at 20:27
  • 4
    Reason #188164828163 to not have password auth enabled on your server. Ever. – EEAA May 24 '12 at 20:32
  • @ErikA: +1 for 'Ever.' – Alex Holst May 24 '12 at 20:37
  • I actually think that VPS providers are doing their customers a disservice by keeping PasswordAuthentication enabled by default. There's no good reason for that. – EEAA May 24 '12 at 20:39

Yes, that is exactly what you do. It is possible that only one user account was compromised, but you'll never know for sure.

You can notify the authorities, but in my experience, nothing happens.

What kind of data do you have? With binary files, you can never be sure, unless you have the same files somewhere safe, and you can compare them. With images, videos, music etc, you're generally safe. With source code... you should check, but since you have git/svn, i'd diff it with a local (uncompromised) copy, and verify the differences are not malicious. I would still check all the configs etc. by hand.


↓ Click the Pretty Cloud! ↓

Nuke it from orbit!

↑ You'll Like It!! ↑

Not the answer you're looking for? Browse other questions tagged or ask your own question.