Today I realized, that my server did not send a logcheck mail, like it should do every night. I logged into my box with ssh and the first commands I typed returned
no disk space left. The administration panel of my hoster (Virtuozzo power panel) displayed a similar warning, 99% used. This was rather strange, because the box is quite new (6 weeks old) and should have a disk usage of 2% (like it said in the logcheck mail the day before). Using
du -sh I tried to find the big file, but there was nothing. No files, no big folders.
After reading about a case where bad session handling caused the disk to fill up in a specific constellation with php, I decided to shut down all of my web applications, one after one (php is not installed on my server). My first guess was my Jenkins server and just after I stopped the service, my administration tool showed
2% disk usage again. So I found the error, but I do not know what caused it. Jenkins was the first thing I installed 6 weeks ago, and there was never a problem until now. I could not see any suspicious access to Jenkins, there were some failed logins by bots in my logs every other day, but I don't this is unusual. I was not able to google this behavior and (of course) am a little worried about it. What can/should I do to prevent it from occuring?
Note that my Jenkins Server currently has one checkin-build and one nightly build, both are small Java-EE applications.