Alternative Linux Detection Methods To Detect A Compromised Host

Question

This is a rewrite of my previous post as it was not entirely clear what I was trying to achieve. Hopefully this will make more sense :)

In a nutshell what I am looking for is alternative methods to detect if my system is compromised beyond using tools such as tripwire/OSSEC/samhain/rkhunter and just general file integrity checking and log monitoring. I am deliberately trying not to be too specific with my question as I am after ideas in general so I can implement them on my own machine. This will probably be but not limited to a script that is ran at set intervals (cronjob) and notifies the administrator if something has changed and prompt an administrator to investigate further via (syslog/email?). Note I am not necessarily after the code itself just a high level overview of what it does. Feel free to be as verbose as you like.

It may help if I list a few things which I currently do to give you a general idea of the ideas I am after.

1) Generate an md5sum of the output of my running iptables and compare it to a known good hash. If this changes it is safe to assume that someone has added/removed an iptables entry.

2) I have certain mount points which are read only (/usr, /boot etc) because they should not change very often. If a partition changes from read only to writable I want to be notified.

3) Monitor the output of netstat for listening services only. Perform a file comparison (diff) against a file containing known good values. If something has been added its possible someone has added a new service to the system. Possible backdoor?

Note the above will generate false positives if I am doing system maintenance. However if these change when I am not I would consider them suspicious and investigate further. Note these are just examples and do have flaws like everything but the more hurdles you have the chances of someone tripping up are increased.

Thanks in advance.

Answer 1

I recommend looking at memory forensics tools, such as those listed on the ForensicsWiki. For example, you can perform kernel and process integrity verification with the Second Look Linux memory forensics product. File integrity checkers are great for validating the non-volatile state of the system; memory forensics lets you validate the volatile state to ensure that all the code running on the system at a given point of time is legitimate and unmodified.

Answer 2

Perhaps I misunderstand your question. It sounds like you just need information on unix host IDS/IPS such as samhain, snort, or OSSEC. There's wiki on it too. Or were you looking for something else?

Answer 3

I run a combination of integrit (binary integrity checking), Tiger (system IDS/auditor), logcheck and seriously harsh IPtables rules (DENY everything I can, including OUTPUT.) all the default settings in Debian work pretty well, performing tasks at regular intervals and emailing output to root.

I'd also recommend the Securing Debian Manual for a good overview of what you can and should do.

Answer 4

I run nmap daily and save the output in xml format.
Something like: nmap 192.168.0.0/24 -oX /tmp/nmap.output

Going through all the options for nmap is to long for here,
but http://www.nmap.org has loads of information.

You can then use ndiff, from nmap packages, to diff the output
from another day and get a list off computers that have been
added, removed or has new ports opened.

Setting all this up in cron and a list is mailed after its run.

Answer 5

OSSEC can do that for you with their system auditing capabilities. Have you tried it out? Look at agentless monitoring of output commands too...

Answer 6

rootkit checks with rkhunter or chkrootkit

Answer 7

You could try Zabbix for services,log monitoring, files checksum integrity and much more. It's quite easy to install and configure . You may need to customize it for your needs adding 3rd party software.

Answer 8

I work on the OSSEC project alot and I know it can solve your problems. Not out of the box, but with a little tuning it will work.

Section 1: Generate an md5sum of ip tables

On of the new features in OSSEC is reporting on the difference of command output to do this just add the following into your etc/ossec.conf

<ossec_config> 
    <localfile> 
        <log_format>command</log_format>
        <command>iptables -L -n</command>
    </localfile>
</ossec_config>

Then create a local rule like the following

<rule id="98989" level="10">
    <if_sid>530</if_sid>
    <match>ossec: output: iptables -L -n</match>
    <check_diff />
    <description>Change made to iptables</description>
</rule>

More details can be found http://www.ossec.net/doc/manual/monitoring/process-monitoring.html

Section 2: have certain mount points which are read only

Currently rootcheck does some simple checks for this, but adding your own are a snap.

First create a rootcheck policy file in etc/shared/ for this example I will name it unix_mount_policy.txt with the following contents.

# This will make sure that the OS this file is used for pilicy is running redhat 
# and it should be changed for your environment 
[Policy - Check OS] [any required] [http://intranet.example.com/location/of/policy]
f:/etc/redhat-release -> r:^Red Hat Enterprise Linux \S+ release 5;
f:/etc/redhat-release -> r:^CentOS && r:release 5.2;

# First real rule
# This will alert if /usr does not have ro on the same line 
[Policy - Mounts read only] [any] [http://intranet.example.com/location/of/policy/readonly]
f:/etc/fstab ->  !r:^# && r:/usr && !r:ro

Section 3: Monitor the output of netstat for listening services only

rootcheck already preforms this function, but if you would like see if anything changes using the common output loggin from my answer in section 1 should be very simple to setup.