Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm running a webserver on CentOS.

Today I couldn't run any SQL queries. Upon investigation, error 28, my /tmp directory couldn't be written to.

I ran du /tmp and it was 1.8Gb. I then ran find /tmp -type f -mtime +8 -exec rm -f {} \; which freed up 300Mb space.

For now, this has done the job - MySQL can operate again.

However, I'm curious as to why the server has so many files in its /tmp directory. I've restarted the server twice and the directory size has remained the same. Don't most linux distributions clear the temp files on reboot?

If I run ls /tmp, I get a massive list of this type of file:

tempCaa7Nw                             tempk7uZU4  temps3GaF1  tempzzWWjV
tempcAbH5o                             tempK7VCdu  temps3hRWv  tempZzXHPM
tempCabZXR                             tempk7VwjK  temps3IdYj  tempzzxZWw
tempcAcQtu                             tempk7W0SI  tempS3IGMF  tempzZy7FM
tempCaCTkx                             tempk7xrzw  tempS3iMfj  tempzZy8L6
tempcAcur7                             tempk7yX3J  tempS3izd5  tempZZYKXr
tempcaCwnz                             tempk81EIE  tempS3JU5s  tempzZyQLO
tempcAdpAk                             tempk81Qpo  tempS3kaNx  tempzZZos7
tempCaeFae                             tempk81ZLV  tempS3KYOx
tempcaEGmt                             tempk83VVy  tempS3LI1u

Full list here.

Do I need to find out what's causing so many temporary files? Or do I simply need to increase the size (or something?) of my /tmp directory? Is that in a config file somewhere?

I'm a poor linux administrator, so please excuse my ignorance.


EDIT for @JeffFerland

Here's the results of grep /tmp on the lsof output file.

mysqld     4649   mysql    4u      REG                7,0         0     103774 /                                          tmp/ib4mpAJ8 (deleted)
mysqld     4649   mysql    5u      REG                7,0         0     103775 /                                          tmp/ibvE7FxN (deleted)
mysqld     4649   mysql    6u      REG                7,0         0     103776 /                                          tmp/ibPe3Lls (deleted)
mysqld     4649   mysql    7u      REG                7,0         0     103777 /                                          tmp/ib656Kb7 (deleted)
mysqld     4649   mysql   11u      REG                7,0         0     103778 /                                          tmp/ibpptQzP (deleted)
cp         7997    root    3r      REG                7,0     18897      28260 /                                          tmp/tempF091j1
cp         7997    root    4w      REG                9,1         0     297690 /                                          tmp_bu/tmp/tempF091j1 (deleted)

ADDITIONAL EDIT Having deleted ~300Mb of files yesterday, my /tmp directory is back to its original size of 1.8Gb, and once again, MySQL isn't working correctly.

Please help!

share|improve this question
    
Are those files or directories? What is in them? –  David Schwartz Feb 8 '13 at 23:08
    
Check the full list @DavidSchwartz; I've listed file sizes etc there too. –  dunc Feb 8 '13 at 23:10
2  
Most of these file have the same size. Check what's in one of them. My first guess would be that your server accepts file uploads and doesn't clean up if you don't use the uploaded files. –  Dennis Kaarsemaker Feb 9 '13 at 8:51
    
@DennisKaarsemaker I'm not sure why I didn't think of that myself. Doh. I've checked about 10 of the tempXXXXX files and they're all just a PHP array() containing loads of settings from my WordPress caching plugin! That gives me a line to investigate, that's for sure. Thank you. –  dunc Feb 9 '13 at 21:37
    
Ahahhhh! Look what I've just found: whatniche.com/2013/02/… –  dunc Feb 9 '13 at 21:39

4 Answers 4

up vote 3 down vote accepted

Most of these file have the same size. Check what's in one of them. My first guess would be that your server accepts file uploads and doesn't clean up if you don't use the uploaded files.

share|improve this answer
    
As per comments on original post, you've hit the nail on the head. All of the tempXXX files were debugging output from an out of date WordPress plugin. They've released a fix; I've updated AND removed all of the unnecessary files - problem solved. –  dunc Feb 9 '13 at 22:49

One thing you could try is using the lsof command to list all of the files currently in use by the OS. That might give you an idea of which running processes are creating/using those /temp.*/ files. From there, assuming you find out which process is the culprit, I would drill down and see what configuration options might be available for reducing the file spam.

share|improve this answer
    
When I run lsof > test there is literally thousands and thousands of processes. How can I work out if any of those are creating temp files? –  dunc Feb 8 '13 at 23:16
2  
@dunc grep /tmp/ test –  Jeff Ferland Feb 8 '13 at 23:23
    
Sorry @JeffFerland, I mean, when I check the test file (I used nano) there are thousands of processes listed in the file created with the lsof command. –  dunc Feb 8 '13 at 23:24
    
'grep /tmp/ test > test2' and use test2 with nano –  Brigo Feb 8 '13 at 23:37
    
Ah, I see. @JeffFerland I've updated my OP with the content of the grep command on my lsof output. –  dunc Feb 9 '13 at 16:36

Different systems do different things. I've worked on systems that did clear out /tmp on reboot (sequent mini's), but most linux distros I've encountered have a cron job tmpwatch that cleans out /tmp.

Some programs create a tmp file and then delete it, keeping the file handle open, so that if the program ends or dies for whatever reason, the space will be reclaimed. You might do

lsof|grep deleted

to see what files are open but deleted. You can then look at /proc//fd and see what files are the biggest and what process has them open.

share|improve this answer
    
He mentioned that after a reboot the space stays the same, so it's unlikely to be deleted files –  kormoc Feb 9 '13 at 1:36

Perhaps it is useful to create a dedicated volume for MySQL, /tmp-mysql, and point MySQL to that path by setting TMP=/tmp-mysql for the start script. That way MySQL should keep operational even when /tmp fills up.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.