0

Possible Duplicate:
How do I know if my Linux server has been hacked?
My server's been hacked EMERGENCY

I am running CentOS 5.3 and here is the result of "chkrootkit": 

Possible t0rn v8 \(or variation\) rootkit installed

Warning: Possible Showtee Rootkit installed
 /usr/include/file.h /usr/include/proc.h
Warning: `//root/.mysql_history' file size is zero
INFECTED (PORTS:  465)
You have    61 process hidden for readdir command
You have    62 process hidden for ps command
chkproc: Warning: Possible LKM Trojan installed
 The tty of the following user process(es) were not found
 in /var/run/utmp !
! RUID          PID TTY    CMD
! root         3040 tty2   /sbin/mingetty tty2
! root         3041 tty3   /sbin/mingetty tty3
! root         3042 tty4   /sbin/mingetty tty4
! root         3043 tty5   /sbin/mingetty tty5
! root         3046 tty6   /sbin/mingetty tty6

I don't understand what the warnings mean.

Is the server infected or in danger?

Edit:

Let me add that I first got strange message on the command line:

Unknown HZ value! (##) Assume 100

Then I followed this great instructions and replaced my hacked files with new ones. I replaced: 

/sbin/ifconfig
/bin/netstat
/usr/bin/pstree
/usr/bin/top

They were all repored as infected by the "chkrootkit".

Now I re-ran "chkrootkit" and got the above output. How to proceed to get rid off all the warnings?

Edit 2:

After checking rpm integrity with: rpm -vVa | grep 'S\.5\.\.\.\.\T' > rpmverify.txt this is what I got:

S.5....T  c /etc/mail/spamassassin/local.cf
S.5....T  c /etc/pam.d/system-auth
S.5....T  c /etc/sudoers
S.5....T  c /etc/samba/smb.conf
S.5....T    /opt/drweb/lib/drweb32.dll
S.5....T    /var/drweb/bases/drw50000.vdb
S.5....T    /var/drweb/bases/drw50001.vdb
S.5....T    /var/drweb/bases/drw50002.vdb
S.5....T    /var/drweb/bases/drw50003.vdb
S.5....T    /var/drweb/bases/drw50004.vdb
S.5....T    /var/drweb/bases/drw50005.vdb
S.5....T    /var/drweb/bases/drw50006.vdb
S.5....T    /var/drweb/bases/drw50007.vdb
S.5....T    /var/drweb/bases/drw50008.vdb
S.5....T    /var/drweb/bases/drw50009.vdb
S.5....T    /var/drweb/bases/drw50010.vdb
S.5....T    /var/drweb/bases/drw50011.vdb
S.5....T    /var/drweb/bases/drw50012.vdb
S.5....T    /var/drweb/bases/drw50013.vdb
S.5....T    /var/drweb/bases/drw50014.vdb
S.5....T    /var/drweb/bases/drw50015.vdb
S.5....T    /var/drweb/bases/drw50016.vdb
S.5....T    /var/drweb/bases/drw50017.vdb
S.5....T    /var/drweb/bases/drw50018.vdb
S.5....T    /var/drweb/bases/drw50019.vdb
S.5....T    /var/drweb/bases/drw50020.vdb
S.5....T    /var/drweb/bases/drw50021.vdb
S.5....T    /var/drweb/bases/drw50022.vdb
S.5....T    /var/drweb/bases/drw50023.vdb
S.5....T    /var/drweb/bases/drw50024.vdb
S.5....T    /var/drweb/bases/drw50025.vdb
S.5....T    /var/drweb/bases/drw50026.vdb
S.5....T    /var/drweb/bases/drw50027.vdb
S.5....T    /var/drweb/bases/drw50028.vdb
S.5....T    /var/drweb/bases/drw50029.vdb
S.5....T    /var/drweb/bases/drwebase.vdb
S.5....T    /var/drweb/bases/drwnasty.vdb
S.5....T    /var/drweb/bases/drwrisky.vdb
S.5....T    /var/drweb/bases/drwtoday.vdb
S.5....T    /var/drweb/bases/dwn50001.vdb
S.5....T    /var/drweb/bases/dwn50002.vdb
S.5....T    /var/drweb/bases/dwntoday.vdb
S.5....T    /var/drweb/bases/dwr50001.vdb
S.5....T    /var/drweb/bases/dwrtoday.vdb
S.5....T    /bin/basename
S.5....T    /bin/cat
S.5....T    /bin/chgrp
S.5....T    /bin/chmod
S.5....T    /bin/chown
S.5....T    /bin/cp
S.5....T    /bin/cut
S.5....T    /bin/dd
S.5....T    /bin/df
S.5....T    /bin/env
S.5....T    /bin/false
S.5....T    /bin/link
S.5....T    /bin/ln
S.5....T  c /etc/proftpd.conf
S.5....T  c /root/.bash_profile
S.5....T  c /etc/httpd/conf.d/mailman.conf
S.5....T    /usr/lib/mailman/Mailman/mm_cfg.pyc
S.5....T  c /etc/drweb/drweb32.ini
S.5....T    /opt/drweb/ldwrap.sh
S.5....T  c /etc/drweb/users.conf
S.5....T    /usr/share/psa-horde/imp/compose.php
S.5....T    /usr/share/psa-horde/imp/contacts.php
S.5....T    /usr/local/psa/admin/plib/api-common/cuMail.php
S.5....T    /usr/local/psa/admin/sbin/autoinstaller
S.5....T    /usr/local/psa/admin/htdocs/modules/watchdog/stats-graph.php
S.5....T    /usr/local/psa/etc/modules/watchdog/monitrc
S.5....T    /usr/local/psa/etc/modules/watchdog/wdcollect.inc.php
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter      /db/backdoorports.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/mirrors.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/programs_bad.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/suspscan.dat
S.5....T  c /etc/courier-imap/imapd.cnf
S.5....T  c /etc/php.ini
S.5....T  c /etc/ssh/sshd_config
S.5....T  c /etc/syslog.conf
S.5....T  c /etc/sysconfig/named
S.5....T  c /etc/httpd/conf.d/ssl.conf
S.5....T  c /etc/smartd.conf
S.5....T  c /etc/vsftpd/vsftpd.conf
S.5....T    /usr/share/psa-horde/util/icon_browser.php
S.5....T  c /etc/init.d/psa
S.5....T    /usr/lib/plesk-9.0/key-handler
S.5....T    /usr/local/psa/admin/htdocs/domains/databases/phpMyAdmin/librari/config.default.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsChecker.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsShow.php
S.5....T    /usr/local/psa/admin/plib/class.RestartServForm.php
S.5....T    /usr/local/psa/admin/plib/class.ServiceControl.php
S.5....T    /usr/local/psa/admin/sbin/packagemng
S.5....T    /usr/local/psa/admin/plib/backup/BackupCreateBackupNowForm.php
S.5....T  c /etc/samba/smbusers
S.5....T  c /etc/pam.d/ekshell
S.5....T  c /etc/pam.d/kshell
S.5....T  c /etc/printcap
S.5....T  c /etc/my.cnf
S.5....T    /usr/bin/spf_example_static
S.5....T    /usr/bin/spfd_static
S.5....T    /usr/bin/spfquery_static
S.5....T    /usr/bin/spftest_static
S.5....T    /usr/lib/libspf2.so.2.1.0
S.5....T  c /etc/awstats/awstats.model.conf
S.5....T    /usr/local/sso/base/Cookie.php
S.5....T  c /etc/httpd/conf/httpd.conf
S.5....T    /usr/sbin/suexec

Does that help?

Edit 3:

Here is rpm check result after core utils have been reinstalled:

S.5....T  c /etc/mail/spamassassin/local.cf
S.5....T  c /etc/pam.d/system-auth
S.5....T  c /etc/sudoers
S.5....T  c /etc/samba/smb.conf
S.5....T    /opt/drweb/lib/drweb32.dll
S.5....T    /var/drweb/bases/drw50000.vdb
S.5....T    /var/drweb/bases/drw50001.vdb
S.5....T    /var/drweb/bases/drw50002.vdb
S.5....T    /var/drweb/bases/drw50003.vdb
S.5....T    /var/drweb/bases/drw50004.vdb
S.5....T    /var/drweb/bases/drw50005.vdb
S.5....T    /var/drweb/bases/drw50006.vdb
S.5....T    /var/drweb/bases/drw50007.vdb
S.5....T    /var/drweb/bases/drw50008.vdb
S.5....T    /var/drweb/bases/drw50009.vdb
S.5....T    /var/drweb/bases/drw50010.vdb
S.5....T    /var/drweb/bases/drw50011.vdb
S.5....T    /var/drweb/bases/drw50012.vdb
S.5....T    /var/drweb/bases/drw50013.vdb
S.5....T    /var/drweb/bases/drw50014.vdb
S.5....T    /var/drweb/bases/drw50015.vdb
S.5....T    /var/drweb/bases/drw50016.vdb
S.5....T    /var/drweb/bases/drw50017.vdb
S.5....T    /var/drweb/bases/drw50018.vdb
S.5....T    /var/drweb/bases/drw50019.vdb
S.5....T    /var/drweb/bases/drw50020.vdb
S.5....T    /var/drweb/bases/drw50021.vdb
S.5....T    /var/drweb/bases/drw50022.vdb
S.5....T    /var/drweb/bases/drw50023.vdb
S.5....T    /var/drweb/bases/drw50024.vdb
S.5....T    /var/drweb/bases/drw50025.vdb
S.5....T    /var/drweb/bases/drw50026.vdb
S.5....T    /var/drweb/bases/drw50027.vdb
S.5....T    /var/drweb/bases/drw50028.vdb
S.5....T    /var/drweb/bases/drw50029.vdb
S.5....T    /var/drweb/bases/drwebase.vdb
S.5....T    /var/drweb/bases/drwnasty.vdb
S.5....T    /var/drweb/bases/drwrisky.vdb
S.5....T    /var/drweb/bases/drwtoday.vdb
S.5....T    /var/drweb/bases/dwn50001.vdb
S.5....T    /var/drweb/bases/dwn50002.vdb
S.5....T    /var/drweb/bases/dwntoday.vdb
S.5....T    /var/drweb/bases/dwr50001.vdb
S.5....T    /var/drweb/bases/dwrtoday.vdb
S.5....T  c /etc/proftpd.conf
S.5....T  c /etc/profile.d/colorls.csh
S.5....T  c /etc/profile.d/colorls.sh
S.5....T  c /root/.bash_profile
S.5....T  c /etc/httpd/conf.d/mailman.conf
S.5....T    /usr/lib/mailman/Mailman/mm_cfg.pyc
S.5....T  c /etc/drweb/drweb32.ini
S.5....T    /opt/drweb/ldwrap.sh
S.5....T  c /etc/drweb/users.conf
S.5....T    /usr/share/psa-horde/imp/compose.php
S.5....T    /usr/share/psa-horde/imp/contacts.php
S.5....T    /usr/local/psa/admin/plib/api-common/cuMail.php
S.5....T    /usr/local/psa/admin/sbin/autoinstaller
S.5....T    /usr/local/psa/admin/htdocs/modules/watchdog/stats-graph.php
S.5....T    /usr/local/psa/etc/modules/watchdog/monitrc
S.5....T    /usr/local/psa/etc/modules/watchdog/wdcollect.inc.php
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter /db/backdoorports.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/mirrors.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/programs_bad.dat
S.5....T    /usr/local/psa/var/modules/watchdog/lib/rkhunter/lib/rkhunter/db/suspscan.dat
S.5....T  c /etc/courier-imap/imapd.cnf
S.5....T  c /etc/php.ini
S.5....T  c /etc/ssh/sshd_config
S.5....T  c /etc/syslog.conf
S.5....T  c /etc/sysconfig/named
S.5....T  c /etc/httpd/conf.d/ssl.conf
S.5....T  c /etc/smartd.conf
S.5....T  c /etc/vsftpd/vsftpd.conf
S.5....T    /usr/share/psa-horde/util/icon_browser.php
S.5....T  c /etc/init.d/psa
S.5....T    /usr/lib/plesk-9.0/key-handler
S.5....T    /usr/local/psa/admin/htdocs/domains/databases/phpMyAdmin/libraries/config.default.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsChecker.php
S.5....T    /usr/local/psa/admin/plib/class.ComponentsShow.php
S.5....T    /usr/local/psa/admin/plib/class.RestartServForm.php
S.5....T    /usr/local/psa/admin/plib/class.ServiceControl.php
S.5....T    /usr/local/psa/admin/sbin/packagemng
S.5....T    /usr/local/psa/admin/plib/backup/BackupCreateBackupNowForm.php
S.5....T  c /etc/samba/smbusers
S.5....T  c /etc/pam.d/ekshell
S.5....T  c /etc/pam.d/kshell
S.5....T  c /etc/printcap
S.5....T  c /etc/my.cnf
S.5....T    /usr/bin/spf_example_static
S.5....T    /usr/bin/spfd_static
S.5....T    /usr/bin/spfquery_static
S.5....T    /usr/bin/spftest_static
S.5....T    /usr/lib/libspf2.so.2.1.0
S.5....T  c /etc/awstats/awstats.model.conf
S.5....T    /usr/local/sso/base/Cookie.php
S.5....T  c /etc/httpd/conf/httpd.conf
S.5....T    /usr/sbin/suexec
Improve this question
9
  • 4
    That means it's time to format the hard disk(s) and reinstall an OS.
    – HopelessN00b
    Oct 4, 2012 at 11:18
  • @HopelessN00b "The only way to be sure!™"
    – Aaron Copley
    Oct 4, 2012 at 11:22
  • @AaronCopley along the same lines as... Which is why it's better to put too many bullets in the head than too few. Overkill >> underkill.
    – HopelessN00b
    Oct 4, 2012 at 11:24
  • 2
    The moment there's a rootkit, there's no way to find out what actually is tained, and what isn't - thats the whole point of a rootkit. Telling you to format, reinstall and restore from backups is constructive. You simply cannot trust that install any more.
    – Journeyman Geek
    Oct 4, 2012 at 11:42
  • 1
    I wouldn't know.. You test your backups lately? If not, it'd be a good idea to do the rebuild on spare hardware. Also, when you rebuild you'll need to make sure you don't reintroduce the same vulnerability. At some level you'll need to know how it got compromised so you can learn from it. Unpatched binaries? Weak password? SQL injection? XSS? What did the server do? You might need to call in some professional help on this if you don't feel comfortable on your own.
    – Aaron Copley
    Oct 4, 2012 at 12:14

1 Answer 1

Reset to default
3

It's a CentOS system. I typically repair these rootkits, but the chances you'll detect/get everything, having not done this before, is slim...

You could start with an RPM verification...

Run rpm -vVa | grep 'S\.5\.\.\.\.\T' > rpmverify.txt

Then examine the output in rpmverify.txt. That would allow you to check to see which binaries and config files to not match the checksums from the RPM database. It's the first place I start on fixing these systems (after making sure no unauthorized network daemons/services are running).

Edit:

I see the output of your RPM verify command. If your yum still works, run yum install yum-utils in order to gain access to the yumdownloader command.

Based on your output, your coreutils and possibly the httpd package has been compromised (cat, df, dd, chown, cp, etc.). Run yumdownloader coreutils to obtain the rpm. It'll download into your current directory. I'd force the reinstallation of the RPM (rpm -ivh --force coreutils*) and re-run the verify I suggested above.

Update:

Hackers/rootkits will often replace binaries with Trojaned versions and set the immutable flag on the file to prevent them from being removed.

Please take a look at the attributes on the /bin/ls binary by running lsattr /bin/ls.

You will possibly see an "a", "u", "i" and "s" in the output. Running chattr -uisa on the same file should remove the immutable flag and allow you to run the rpm installation.

The attributes should look like:

[root@kitteh ~]# lsattr /bin/ls
------------- /bin/ls

Repeat for any other files that fail in the RPM installation. You may need to also change/remove those attributes on the enclosing directory...

Improve this answer
13
  • If you're going to do this, you should do it from a rescue environment (otherwise you can't trust the results.)
    – Aaron Copley
    Oct 4, 2012 at 12:03
  • This is true, although rootkits rarely target RPM itself (even though they should). It's a good quick way to get an assessment.
    – ewwhite
    Oct 4, 2012 at 12:13
  • I ran the command, please see Edit 2 added above.
    – Danijel
    Oct 4, 2012 at 12:16
  • 2
    @Danijel This is really beyond the scope of what I can provide in a Server Fault answer. I was just trying to help point you in the right direction.
    – ewwhite
    Oct 4, 2012 at 13:47
  • 1
    @Danijel well, it still can be helpful as a closed question, and by linking it to two other similar questions, I'd argue it's made more useful, because it now contains easily accessible linkage to two similar questions, even if they are more generic. A closed question isn't necessarily a bad thing. I have one myself, and am actually rather proud of it, and thinks it makes a contribution as well.
    – HopelessN00b
    Oct 4, 2012 at 15:13

Not the answer you're looking for? Browse other questions tagged .