Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

My ISP has contacted me stating that unauthorized attacks are coming from my LAMP server. Here is part of the log they have attached:

15:26:16.821245 IP My.Ip.is.here.59987 > some.one.elses.ip.ssh: UDP, length 1
15:26:16.821248 IP My.Ip.is.here.59987 > some.one.elses.ip.ssh: UDP, length 1
15:26:16.821251 IP My.Ip.is.here.59987 > some.one.elses.ip.ssh: UDP, length 1
15:26:16.821253 IP My.Ip.is.here.59987 > some.one.elses.ip.ssh: UDP, length 1 

How do I investigate this?

share|improve this question
    
Is this SSH over UDP!? You need to monitor the traffic generated from your server to know what process is doing this. –  Khaled Dec 15 '10 at 16:23
    
@Khaled - Exactly. How do I do this? –  Ra. Dec 15 '10 at 17:10
add comment

4 Answers

First, either shut down or disconnect your server from the network. That will stop any attacks that are still in progress. It's very likely that your server is compromised. You'll need to do a full re-install and restore from a known-good backup. Yes, that's a pain, but it's the only foolproof way to ensure that you're restoring "clean" files. Before doing that, either swap out the hard drives of the server or take an image of them. That will let you do some forensic work to figure out what happens.

Once the new server is up, it goes without saying that you'll need to ensure that all passwords are changed to something long and complex or preferrably, just disable password auth and use key auth instead.

share|improve this answer
add comment

Run a netstat -anp | grep 22 and take note of the PIDs listed on the right. You can then lsof -p PID to see which processes are running the ssh scans. (you may also want to redirect that output to a file to have a record of where the intruder's scripts are running) Kill those processes to start, but then you have to begin the job of finding backdoors and anything that may reactivate those scanning processes (via cron or otherwise). Try one of the usual rootkit hunting utilities like chkrootkit or rkhunter.

share|improve this answer
    
netstat -anp | grep 22 will show if someone is connected to my server via SSH, right? What I need to know is whether a script on my server tries to connect to someone else's server via SSH. –  Ra. Dec 15 '10 at 16:45
    
Yes. It would show your incoming and outgoing connections. There are multiple colums indicating the local address:port and the foreign address:port. If you are logged in to your system, try looking through the "last" log to see where recent logins came from. –  ewwhite Dec 15 '10 at 16:49
    
Yep, "last" was the first thing I tried and no one has ever logged to my server since I did a full re-install a week ago. –  Ra. Dec 15 '10 at 17:07
    
Perhaps grep -i "Accepted pass" /var/log/secure* –  ewwhite Dec 15 '10 at 17:19
add comment

Use lsof or netstat to see what is connected to that port on your system. Use ps to find the process tree to see where it came from. Patch the affected application(s).

share|improve this answer
add comment

It's quite possible for programs to run while their behaviour is hidden from user-space commands. Also that the attacker may have installed additional programs and backdoors. And you've not found out how your machine was originally compromised.

i.e. even if you find the origin of this traffic, your machine is not clean, nor is it secure against future compromises.

Even assuming you are just being a bit coy and are a real computer genius who can track down those responsible - so what? Do you think its likely that you'll be able to extradite those responsible and get them convicted.

By all means treat it as a learning exercise - but don't expect to gain anything from this other than a bit of knowledge.

Time to start thinking about how you clean the system and get it back online securely - and how you prevent / detect future attacks.

share|improve this answer
    
Who said anything about extradition or criminal charges? Realizing a box is compromised is stressful enough as is without the condescending tone and the cliche "learn from your mistakes and get your machine back online securely" line that I see in almost every one of these posts where people ask for help with a compromised machine. –  DKNUCKLES Dec 15 '10 at 18:44
    
@DKNUCKLES - the responsible thing to do is exactly that...take the machine down, optionally image the drive, and do a clean restore. Leaving the system up is just asking for problems and without doing a re-install, you're never going to be 100% certain you have a clean system to work with. –  EEAA Dec 15 '10 at 20:43
    
@ErikA - I know that's the correct thing to do however the question was "how to investigate the matter" and not necessarily how to clean the server. Besides that, symcbean's answer was quite demeaning and rude. –  DKNUCKLES Dec 15 '10 at 21:26
    
-1 for answering in bad faith and/or poor use of snark. Who said anything about tracking down who was responsible? lsof -p PID will yield useful info about how it was compromised. I'm sure people will want a post mortem. –  JamesBarnett Dec 31 '10 at 9:32
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.