Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an EC2 instance that may have been compromised. The instance responds far too slowly as of last week and recently I've checked the httpd access_logs and they are in excess of 11Gigs, despite opening the EC2 instance only last month. Today's access_log is growing. The log file is full of sites that have nothing to do with the site I am working on.

A handful of people had ssh access, but I've revoked it. Still, the log files grow.

Ended up getting some emails from amazon about "abuse" and found that there was a jump in mail activity. Strangely, I never set up mail service on this instance but the firewall has been set to allow smtp access (amongst other things), this is even though I only set port 80 access and a few port 22's.

I'm not really much of a server guy and I don't know any.

I've changed my Amazon passwords and created new access codes. Changed the ports for ssh to different ones.

I'd really appreciate any advice on this subject. Thanks.

//** EDIT ****/

I now believe that this was the result of having my server used as a proxy. As per these guidelines http://wiki.apache.org/httpd/ProxyAbuse I've managed to stem a great majority of the false traffic going to the site, though the requests are still piling up on the access_logs. I've confirmed by trying to use my server as a proxy and it forces users to the site directly -- the way I guess it should -- instead of say, yahoo.com.

I don't know if this was the solution but so far it seems to be working out -- so far.
Still, the answers posted have opened my eyes. If anyone has more to tell I would gladly hear it. Thanks very much!

share|improve this question

3 Answers 3

There could be one of two ways the server is being abused or got broken into. First via a software exploit and second via the web application route. I would first check that for software exploits and then check the web application.

Possible problems. In no particular order

  1. Do you have any mailing/form which uses user input to send emails and doesn't have authentication/authorization involved.
  2. Misconfiguration or unsecured applications. Things like phpMyadmin without password or in development applications exposed in an unsecure way.
  3. Check the access logs and get a list of files that were accessed. If they are your files I would check for security holes in them.
  4. Check your known good backup with current files present on your web server. If you didn't use any secure coding guidelines the there could be a problem with Remote Shell etc.

If you are not using mailing block smtp at the firewall to contain the damage.

share|improve this answer
    
Thanks Sameer. I've blocked smtp and indeed most of the incoming ports except http for the moment. I'm checking on the others right now. –  sandrum Apr 19 '11 at 3:17

Look through the pages you serve, look for anything like spammy links or exploit code that has been inserted into your server. Your best bet is to compare your static files with a backup and see if anything has been altered.

Close port 25 at the firewall if you aren't using it.

You can use tools like rkhunter:

http://www.rootkit.nl/projects/rootkit_hunter.html

and chkrootkit:

http://www.chkrootkit.org/

to try and detect tools that hackers use to maintain control over your system.

Use a tool like Nexpose to find vulnerabilities in your web application and OS and follow their recommendations on remediating those vulnerabilities:

http://www.rapid7.com/vulnerability-scanner.jsp

share|improve this answer
    
Thanks AB. I've tried these above but found nothing out of the ordinary. Still, it's good to have these in the arsenal. I've taken note. Many thanks. –  sandrum Apr 19 '11 at 3:12
    
Note, a negative result from these tools doesn't guaranteed there's no breach. If they've obtained root access, a sufficiently sophisticated hacker could hide their tools or replace these tools with versions that return dummy reports. The only guaranteed method is to completely reformat and reinstall the OS. –  Cerin Feb 25 '13 at 15:59

If requests are still coming in, change the elastic IP address. Often public proxies are referred to by IP address, this should stop the remaining traffic.

It's worth checking any IP address assigned to you by Googling it when you get it. I had one that was listed as a proxy, so just got another.

share|improve this answer
    
Upon allocating a new IP address, I've searched to find that indeed it's not known as a proxy IP so I think I'll do that. Thanks David. –  sandrum Apr 19 '11 at 3:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.