0

The Ubuntu Web server that I just inherited has been compromised. I'm a Windows admin and not all too familiar with Linux.

How would I go about finding out what keeps spawning perl processes? I killed a couple dozen instances of /usr/bin/perl but they are spawned again immediately. I've even renamed /usr/bin/perl but the processes still come back.

I recognize that I'll have to nuke the server, but I need to set up a replacement before I can take the website offline.

4 Answers 4

2

If you're sure it's hacked and this isn't a misbehaving or normal behaving for that matter service or cron job on the server then the first thing you need to do is get it off the net. Put a quick maintenance page or something notifying customers/users that the site is down for maintenance.

You aren't doing yourself any favors if your site gets tagged by Google, Microsoft, and AV companies as serving malicious content. Or if it's sending spam or being used as a stepping stone to attack other servers.

Make an image of the system and then nuke and rebuild to get it back on line as quick as possible. Now that the fire is out you can now focus on using that image and perhaps even restoring the image to a VM to do your forensics.

2
  • 2
    +1 for getting this box off the net. If you're reasonably certain you've been compromised "down for maintenance" is almost certainly better than "up, harvesting your credit card numbers and sending you viagra spam"...
    – voretaq7
    Jun 22, 2010 at 21:02
  • Thanks for the recommendation -- I've taken it down. I'd still like to find out what's going on before I put a potentially vulnerable web app back online.
    – user46497
    Jun 22, 2010 at 21:23
0

What's the parent process ID of those new processes? Use ps -lp <PID>. The listing will have a column, PPID, which includes the process ID of the program that launched it.

Edit How about getting more information the hard way. Go to /proc//, and check the values of some of the soft links there. For example, "cwd" will point to the working directory of the script, and "exe" will point to the executable involved. That may give you some useful information. Alternately, attach strace to the process involved via strace -p <pid>, and see what calls it's making.


--Christopher Karel

2
  • PPID is 1 (init)
    – user46497
    Jun 22, 2010 at 21:25
  • D'oh. Edited with some more in depth stuff. Try that. Jun 22, 2010 at 21:45
0

Try chkrootkit, worked for me decades ago.
The other option was rkhunter.

These two can help you identify files added to you OS and delete, deactivate some of them. I used both to do some forensics about how the system was modified after a service was exploited.

HTH

0

check /etc/inittab for anything suspect:

0:2345:once:/usr/sbin/ttyload

would be one such script. You may have removed/renamed /usr/bin/perl, but, Debian/Ubuntu will also have a /usr/bin/perl5.10.1 (or whatever your current version you have installed). Perl4 may also still be on your achine.

1
  • I did check inittab. Nothing jumped out at me as suspect, but then again I wouldn't necessarily recognize something that is out of place. Based on a previous suggestion I checked out /proc for one of the perl processes, and the command line pointed to my /usr/bin/perl-remove (which didn't have the executable bits set).
    – user46497
    Jun 24, 2010 at 13:21

You must log in to answer this question.