I have an ubuntu server running in my bedroom. It's connected to the internet. Last night, at 5am, it was doing some intensive i/o with the hard drive (I heard it) for like 20 minutes. I don't have any cron jobs scheduled, and it has not done that before. Could it have been hacked? Or am I being paranoid... Was the ext4 filesystem doing some journaling update? What would you do to gather more information next time it may happen. Perhaps an IDS or ... ? It was behind a firewall.
2 Answers
Many Linux distros run some daily cron jobs early in the morning. Indexing the harddrive for slocate and log rotation will both produce intensive disk activity for a few minutes depending on the size of your harddisk.
I'm not sure if Ubuntu is configured the same way, but Redhat based distros usually have an /etc/cron.daily file or directory. Read up on cron and how it works and look at the files in that directory.
In my experience most hacked systems are used for their network connection, therefore intensive disk activity is not a great indicator of malicious activity.
Unless you intentionally removed cron then you do have cron jobs scheduled. Several are set up by various packages as part of their normal operation. You can find them in /etc/crontab, /etc/cron.d/, /etc/cron.daily/, /etc/cron.weekly/, and /etc/cron.monthly/.
That sounds to me like updatedb was running. updatedb finds every file on your server and indexes them for locate to use to, well, locate files quickly. Crawling through the entire filesystem can take a while depending on what you have on there and it will keep the disk busy for that entire time.
/var/log/syslog?