Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have been seeing a recurring issue on my web server:

[Sun May 16 03:10:19 2010] [crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock
Configuration Failed
[Sun May 16 04:10:05 2010] [crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock
Configuration Failed
[Sun May 16 05:10:04 2010] [crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock
Configuration Failed
[Sun May 16 05:17:13 2010] [crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock
Configuration Failed

So far, the only fix I have found to this when it happens is to reboot my server. This is non-ideal :-\

Restarting httpd does not clear the error. df indicates I have 20+ gigs free, and top and free both report 800+ megs (or 1.2 gigs)

> df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/simfs             40G   18G   23G  44% /

#

> free
             total       used       free     shared    buffers     cached
Mem:       1474560     300832    1173728          0          0          0
-/+ buffers/cache:     300832    1173728

Any ideas on why this would recur, and how to prevent/fix it?

share|improve this question

2 Answers 2

up vote 3 down vote accepted

http://devtime.blogspot.com/2008/02/apache-error-no-space-left-on-device.html

Apache Error: No space left on device

Apache doesn't start and the error log contains:

[emerg] (28)No space left on device: Couldn't create accept lock

or

[crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock Configuration Failed

After checking your disk it shows that you have plenty of space. The problem is that apache didn't shut down properly, and it's left myriads of semaphore-arrays left, owned by 'apache' user.

Run:

ipcs -s | grep apache

Removing these semaphores immediately should solve the problem and allow apache to start.

ipcs -s | grep apache | perl -e 'while () { @a=split(/\s+/); print `ipcrm sem $a[1]`}'

where apache is the owner of the apache process, if that's not the name of the user on your server change it.

share|improve this answer
    
thanks. Could you copy/paste the scriptlet to your answer for posterity? –  warren May 17 '10 at 13:14
    
I needed to modify the scriptlet on that page to the following: ipcs -s | grep apache | cut -b11-22 | xargs ipcrm -s –  warren May 17 '10 at 15:11

That xargs didn't seem to work for me, so in bash I did this instead:

for x in ipcs -s | grep apache | cut -b11-22; do ipcrm -s $x; done

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.