Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

My Ubuntu 10.04 server is down since this morning. When I restart it, I can use it for a couple of minutes then the RAM and Swap get full, and SSH doesn't answer. It seems ClamAV is part of the problem. How can I stop it? clamd stop didn't work.
And more globally, how can I stop all email services?

UPDATE: I don't think log files are needed anymore.

running processes

UPDATE 2:

wpc2617:~# free -m
                total    used    free  shared    buffers     cached
Mem:           875     868       7       0          0         12
-/+ buffers/cache:     855      20
Swap:          976     867     108
wpc2617:~# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 1 31 971132   8076    696  18988  311 1300  2915  1309  244  200  9  2 11 77
share|improve this question
    
In this screenshot I stopped postfix and the server is still extremely slow. It seems caused by clamscan, but I tried "clamd stop" and it didn't change anything. Apache and MySQL won't start and everything slown down big time –  Nabab May 4 '11 at 14:22
1  
try killall clamscan then stop whatever is starting those processes. The "D" under the S column means the processes are in "uninterruptible sleep" which could be due to various reasons but is probably due to paging to/from swap (the top also shows all your swap is used.) Killing the processes may be a challenge - see this answer - so you may need to reconfigure system not to use clamscan, then restart server. –  RichVel May 4 '11 at 14:40
    
Thanks a million for killall, that's the command I was looking for. I've been able to kill all clamscan, stop postfix, then after uninstalling/reinstalling clamav everything was working fine. –  Nabab May 4 '11 at 18:45
add comment

4 Answers

up vote 1 down vote accepted

(Converting my comment on question into an answer as it seemed to work - hope that's OK.)

Try killall clamscan, then stop whatever is starting those processes. The "D" under the S column means the processes are in "uninterruptible sleep" which could be due to various reasons but is probably due to paging to/from swap as your memory and swap space are entirely full (from the top and free -m "buffer/cache" line).

Killing the processes may be a challenge (see this answer on uninterruptible processes), so you may need to reconfigure system not to use clamscan, then restart server.

share|improve this answer
add comment

A few comments and things you can check:

  • I don't see anything obviously wrong in the log you posted. What makes you think the last line in the log is an issue?
  • Near 100% RAM utilization is actually a good thing in general. What you don't want to happen, though, is begin to use swap space which will kill your server's performance pretty quickly and may be the issue hear. Use the top command and ensure that the swap memory is near zero. free -m can also be used to see how your memory is in use along with vmstat.
  • Also check your CPU usage with top. If the usage or loads are high that could be another reason.
  • If you are being DoS you also may have a fully used network connection (especially on a 10 Mbps line). Check your network usage with ifconfig or your provider's control panel.
  • If you cannot seem to catch the issue in real time try installing a basic monitoring package like sar to capture stats that you can given later.
share|improve this answer
1  
Kind of depends: near 100% with cache is good, without is not :). If orignal poster does free -m, he/she should look at the "-/+ cache" column. –  Halfgaar May 4 '11 at 12:40
1  
@Halfgaar It's a row, not a column. –  BMDan May 4 '11 at 13:29
    
Thanks, yeah the log was irrelevant in fact. So I have a full swap, I've added the result of free -m and vmstat. I don't know how to use ifconfig but I'd like to find out first how to stop all email services –  Nabab May 4 '11 at 14:52
    
BMDan: whoops... –  Halfgaar May 5 '11 at 8:04
add comment

try starting the server and then immediately log in and run top then press M (being actually a capital M to sort by memory usage) and watch your resources. You should see your swap at 0 bytes used and your memory free either fully used as linux can use sometimes or little under but mainly watch the top of the light of processes consuming memory. You should see the pattern of what memory starts to increase if an app has a memory leak. if you see what it is, stop the process/service and see how it goes after 30min if it lasts.

good luck

EDIT: the problem there is that your mail logs will show that amavis cannot find any primary scanners and is most likely fallen back to secondary scanning which in your case is clamscan. clamscan is very slow and starts a process for each mail or file it scans. What you need is clamd and configure amavisd to use it as a primary scanner. If you don't have amavisd being used (though you should), then you are filtering using procmail as this appears to be running as the user account. You need to revise your setup and possibly configure postfix to connect to an amavisd service to best handle scanning.

share|improve this answer
    
Thanks, I'll try that as soon as the guys at the hosting company get out of the recovery mode –  Nabab May 4 '11 at 11:43
    
So the swap is fully used, that's bad isn't it? –  Nabab May 4 '11 at 14:26
add comment
# To free pagecache:
#         echo 1 > /proc/sys/vm/drop_caches
# To free dentries and inodes:
#         echo 2 > /proc/sys/vm/drop_caches
# To free pagecache, dentries and inodes:
#         echo 3 > /proc/sys/vm/drop_caches
sudo sync && sudo sysctl -w vm.drop_caches=3
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.