1

So I got hit by a script kitte... Fortunately the box is Ubuntu and was able to replace w/ binaries from a comparable system, however,

Some of the files I couldn't delete, and am still stumped on this. The hijacked files are sitting in the /_bin directory which is writeable by root.

nathan@db-0:~$ ls -ld !$
ls -ld /_bin
drwxr-xr-x 2 root root 4096 Mar 12 18:00 /_bin

Ok, those are the perms on the directory, now for the files within:

nathan@db-0:~$ ls -l /_bin
total 268
-rwxr-xr-x 1 root root  39696 Nov 19 22:25 ls
-rwxr-xr-x 1 root root 119800 Mar 31  2012 netstat
-rwxr-xr-x 1 root root 101240 Dec 12  2011 ps

Now when I try to delete one of these files (as root):

root@db-0:/home/nathan# rm /_bin/ls
rm: cannot remove `/_bin/ls': Operation not permitted

Or if I try to delete the entire _bin directory (again as root):

root@db-0:/home/nathan# rm -rf /_bin
rm: cannot remove `/_bin/ls': Operation not permitted
rm: cannot remove `/_bin/netstat': Operation not permitted
rm: cannot remove `/_bin/ps': Operation not permitted

So how can I delete these files?

Edit:

Sure enough the immutable bit has been set, however, removing it does not let me delete the files.

root@db-0:/home/nathan# lsattr /_bin
s---ia--------- /_bin/ls
s---ia--------- /_bin/netstat
s---ia--------- /_bin/ps

root@db-0:/home/nathan# chattr -R -i /_bin
root@db-0:/home/nathan# lsattr /_bin
s----a--------- /_bin/ls
s----a--------- /_bin/netstat
s----a--------- /_bin/ps

root@db-0:/home/nathan# rm -rf /_bin
rm: cannot remove `/_bin/ls': Operation not permitted
rm: cannot remove `/_bin/netstat': Operation not permitted
rm: cannot remove `/_bin/ps': Operation not permitted

Also verified /_bin doesn't have immutable bit:

root@db-0:/home/nathan# lsattr -d /_bin
--------------- /_bin
6
  • 7
    Reinstall the system from scratch, don't try to repair it. You'll be never sure to have eliminated everything.
    – Sven
    Mar 12, 2013 at 18:16
  • Can you do strace -v rm /_bin/ls and as it's longer output, put it on the web like pastebin.com?
    – ott--
    Mar 12, 2013 at 18:34
  • 2
    I say we take off and nuke the entire site from orbit. It's the only way to be sure. youtube.com/watch?v=aCbfMkh940Q Mar 12, 2013 at 19:05
  • The answer you accepted says there are other extended attributes that might prevent removal... there might also be /_bin/.* files with funky attributes.
    – vonbrand
    Mar 12, 2013 at 22:14
  • Good man @vonbrand, though that wasn't the case here an old colleague once told me of a case where there was a ... directory - tricky kitty! Mar 13, 2013 at 0:06

2 Answers 2

10

Most likely the attacker has set the immutable attribute on the files and directory. This is commonly done by rootkits to make cleanup more difficult.

To confirm this, try:

lsattr /_bin

To remove the immutable attribute, use:

chattr -R -i /_bin

You'll also want to clear the a and s attributes, since these may affect your ability to remove the files.

chattr -R -i -a -s /_bin

See the chattr man page for a full explanation of what all the attributes are and what they do.

0
-1

Looks like the sticky bit is still there.

chmod -t /_bin
1
  • That is irrelevant...
    – vonbrand
    Mar 12, 2013 at 22:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .