up vote 1 down vote favorite
1
share [g+] share [fb]

Woke up this morning to find out MongoDB grew to all available disk space for some reason. I cannot repair the db to compact it as i have no space left. I can't even start postgres to remove an old database which would free up a few gigs more. I literally have zero free disk space.

I did delete a 9GB file but the space is not showing up at all when i "df -h".

I have to clear up space on the drive asap as its a production server and we're down. Can anyone help?

Thanks!!!!

Harel

link|improve this question
78% accept rate
feedback

2 Answers

up vote 6 down vote

the space will not show up until all processes release the file. Try using lsof to see what's using the file.

link|improve this answer
The file i deleted was an old db sql dump. It was not used by anything. – Harel Mar 21 '11 at 11:44
To make things worst, i droped the bad mongodb collection but i cannot run repairDatabase which would reclaim the space, because I don't have any space left! Its a catch22 – Harel Mar 21 '11 at 11:45
Did you double check lsof to see if there are any open file handles? – Bart Silverstrim Mar 21 '11 at 12:22
Yes. I also stopped all the services I could stop. No matter what I deleted, df -h shows 0 bytes left on drive (even after i run 'sync' or reboot). – Harel Mar 21 '11 at 12:24
feedback
up vote 1 down vote accepted

Although the answer provided is valid in many cases, in my scenario I ended up falling into the ext3 5% reserved root space. I could have recovered this space had I known about it at the time. I ended up using a USB drive as a repair path to mongodb reclaiming back 850GB.

This is how to recover that ext3 5% should anyone be interested:

http://www.andremiller.net/content/recovering-reserved-space-ext2-and-ext3-filesystems

Thanks for all the comments!

Harel

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.