-2

I run my python scripts and Scrapy framework for web scraping project on my Ubuntu 12.04 precise server. These scripts run whole day.

This project is under developing/testing stage. So i dont know what will be the system requirement of this project.

I started it with 512MB RAM and 30GB Hard Disk. The system crashed because of less hard disk space. so once again i have to setup the server and restarted my project with 512MB RAM and 100GB Hard Disk.

Again the system crashed because of less RAM/SWAP size.

Is there a way to apply checks on RAM, Hard Disk ? So that beforeserver crash it will kill all the programs and send me mail that all programs are killed because of less space increase your RAM/HARD DISK size.


I am looking for a shell script which is handled by cronjob running in every minute to check whether disk is full.

3
  • your above mentioned question dose not answer my question. It is very general. Basically i am looking for a shell script to do this task Aug 21, 2013 at 8:39
  • 1
    @binit: You need to write that script for yourself then and base the actions it takes on your knowledge of how your system performs.
    – user9517
    Aug 21, 2013 at 9:33
  • 1
    @binit Server fault is not going to "give you teh c0dez" -- we can't, because we don't know your environment. You can certainly write an appropriate shell script (or have your monitoring system take appropriate action), and someone here even gave you a good starting point, but final implementation in your environment is up to you.
    – voretaq7
    Aug 21, 2013 at 13:49

3 Answers 3

2

Monitoring

AWS has a service called CloudWatch that can monitor the health of your system. http://aws.amazon.com/cloudwatch/

You could script things like "killing programs and sending emails" yourself based off cloudwatch metrics.

Of course there are plenty of other tools to monitor the health of your system such as:

But monitoring isn't the only thing you can do!

There's also plenty of other things you could do.

You could store your data in AWS S3. Then you wouldn't run out of disk space.

You could use multiple EC2 instances, in an auto scale group, or in a number of other configurations that allow them to share the load.

AWS have a number of database services that are managed and can scale for you.

4
  • But i am using rackspace server is there any facility too. Aug 21, 2013 at 6:36
  • My answer gives you general information because you have asked a general question. If you want better answers you need to ask better questions. I've already provided solutions that work on AWS and other providers too. Aug 21, 2013 at 6:39
  • Answers you suggested is costly. A shell script which performs this task can also be written. Aug 21, 2013 at 7:12
  • 5
    Then feel free to write it. Aug 21, 2013 at 7:26
1

The Monit tool can do what you need -- monitor RAM and disk usage, kill offending process and alert you.

1

Actually, it seems like you're looking for an "easy and brute" solution and that could be something similar to:

#!/bin/bash

partition="/dev/sda1";
minRAM=512
minDisk=1024
maxSwapUsed=1024

freeMem=$(free -m | grep Mem | awk {'print $4'});
usedSwap=$(free -m | grep Swap | awk {'print $3'});
freeDisk=$(df -m  | grep $partition | awk {'print $4'});

echo "RAM free = " $freeMem;
echo "Swap used = " $usedSwap;
echo "Disk free = " $freeDisk;

if [ "$freeMem" -gt "$minRAM" ] && [ "$freeDisk" -gt "$minDisk" ] && [ "$usedSwap" -lt "$maxSwap" ]; then {
    echo  "Everything is ok";
} else {
    echo "Kill da pid";
fi

Add it to your crontab, setup mins/max/partition and which is the action to take in pkill. Anyway nagios / snmpchecks would have been more proper and obviously more complex and precise.

1
  • Wops, there was an error in swapcheck :/ Aug 21, 2013 at 13:21

Not the answer you're looking for? Browse other questions tagged .