My server's been hacked EMERGENCY
First some background info. The server we're talking about is running CentOS 5.6, SSH on port 22 which can be accessed over the Internet (bad, we know), Apache on port 8080 which can be accessed over the Internet and MySQL, which can't be accessed over the Internet. A few days ago this testserver was hacked, due to a really simple password (yep, you'll find it all over the Internet, bad...), so we changed it. Of course, this time we entered a real password, something you wouldn't find on the Internet.
That far, everyone went okay. We had scanned for rootkits, deleted some stuff the script kiddies had downloaded (they logged in using SSH), well, kinda the usual things you'd do. However, today we saw that the hacked server was using so much traffic, that we just had to check if everything was OK. So, we logged in using SSH, and what did we see? Yep: "Last login 21 Feb 22:08 from xxx.xxx.xxx.xxx". And yep, same IP as the one that was used to log in to the system before changing the passwords.
However, this time those script kiddies installed some malware. They apparently created some accounts, and one was called oracle. It contained the folder .mozilla which contained the folder lala. Opening that folder, we saw the same stuff as the script kiddies had downloaded with the really-unsafe-password account.. Trying to do a ps -x, we saw the process zmeu running. And not just 1, think of like a 50 or so. Now come some questions :-)
How did those script kiddies log in? No SSH keys were uploaded and the passwords were changed. However, they did manage to log in, after all...
What does 'zmeu' do? Looks like it has something to do with phpmyadmin, but we don't use phpmyadmin. We've stopped all 'zmeu' processes, deleted the files and accounts and rebooted the server.
The network totally dropped. Everything was slow as hell, as long as this hacked server was connected to the network. Looks like it has been fixed by deleting the zmeu processes.