Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

One of our web-projects was hacked. Malefactor changed some template files in project and 1 core file of the web-framework (it's one of the famous php-frameworks). We found all corrupted files by git and reverted them. So now I need to find the weak point.

With high probability we can say, that it's not the ftp or ssh password abduction. The support specialist of hosting provider (after logs analysis) said that it was the security hole in our code.

My questions:

1) What tools should I use, to review access and error logs of Apache? (Our server distro is Debian).

2) Can you write tips of suspicious lines detection in logs? Maybe tutorials or primers of some useful regexps or techniques?

3) How to separate "normal user behavior" from suspicious in logs.

4) Is there any way to preventing attacks in Apache?

Thanks for your help.

share|improve this question
    
Found a good article: How to Tell if Your PHP Site has been Hacked or Compromised (gregfreeman.org/2013/…) –  Vasiliy Toporov Aug 4 '13 at 4:42

5 Answers 5

Like HopelessNOOb, I'd recomend getting some professional help with this.

When you know that a system has been compromised you can't rely on any data stored on it. Further, if the compromise was via HTTP, then there's probably not enough information in the standard logs to be able to isolate what hapenned. This is why people who are serious about webserver security will use something like mod_security to get much more detailed logs, and run a host based IDS to detect compromises.

The support specialist of hosting provider (after logs analysis) said that it was the security hole in our code

If he/she can make that claim then he/she must already know the answer to your question - either that or the claim is not substantiated and they really mean to say that they couldn't find anything wrong with their stuff.

Your objective should be to get your site back online and secure. To make a site secure, you need to plug every hole - but to compromise a site you only need to find one hole: you know you have at least one hole in your site, but you need to fix any which exist - which means you need to take a very different approach to your site security. Even of you can identify and fix the vulnerability which was exploited this time, the evidence shows that you don't have the processes and skills in place to have any sort of confidence that this was an isolated incident.

share|improve this answer

1) You can have a look at these links for some software to help you, though I've not used most of them myself. But given that you've got timestamps on the files that were altered, start by looking at the log files for those dates and time. Grep for the filename of one of the files altered; that should give you the ip address of the malefactor, and then you can start grepping for that.

2 and 3) The first thing you need to do is to find out what is normal for your own site. One way to do that is to have a standard analyzing tool, such as awstat, go through your logs daily. Your main problem now appears to be that you don't know what is normal for your site, so you have no way of knowing what is abnormal.

4) There are ways of preventing some attacks, though of course it's an arms race. But just by removing the most common exploits you'll make it far less likely that you'll be the target of a random attack. You should make sure that your installation has the latest security patches for your OS, for apache and for php. You should also have a look at mod_security, which allows you to both log suspicious activity and to e.g. require that a client send headers such as User-Agent, which many crack tools don't.

Also, I'd advise you to consider sending your logs, especially error logs and any mod_security logs, to another server if possible, so that even if somebody manages to get into your server they won't be able to edit the log files to hide their tracks.

share|improve this answer

You got a good answer from JennyD, but I'd suggest a different approach.

Hire an expert.

There are security experts who actually do this for a living and will do a far better, far quicker job of determining how you were hacked and where you're still vulnerable. Check out the local security market, and I'd be surprised if you can't find a decent price on a consulting services to analyze the logs for you, as well as run at least a basic penetration and vulnerability scan on your system(s). You'll get a much more complete picture of what you need to do to secure yourself, and you'll be able to get back to spending your time within your areas of expertise.

share|improve this answer

In addition to the guidance here, these topics are discussed in depth over on the security stack exchange. Have a read of the question on Apache hardening: http://security.stackexchange.com/questions/77/485

share|improve this answer

You might want to check out LORG - https://github.com/jensvoid/lorg - a PHP-CLI tool (GPL) designed for post-attack forenscis of Apache access_logs. It uses several methods (signature-based, statistics, learning-based) to detect attacks against web applications in HTTPD log files.

It tries to group those suspicious activities into sessions and applies robot detection techniques (typically, an attacker might run an automated scan and then come back later to manually check what it has found). Also, it uses geomapping and DNSBL-lookups to see if the attacks orignate from a certain country or botnet.

Afterwards you can choose various mechanisms to determine if an attack was successful: active replays and matching for signatures, HTTP response codes or outlyingness in the size of responses (the bytes-sent field).

Obviously, if the log files have been altered or deleted, things might get difficult (LORG has the option to perform a simple tamper check to find extraordinary time windows with no activity, but this won't really help you).

As for preventing attacks, i'd go with mod_security (if yourhosting provider allows it).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.