Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

What I basically want is to to write all tcpdump captured packets to a file every 3 days. So basically tcpdump should run for 24 hours on day 1 and write the output to Day1.log and similar for Day2 and Day3. On the 4th day it should repeat and write the log to Day1 again. This is basically to check DDoS attempts on my server and to find out the type of attack including the attacker's IP as in the last 7 days my machines were DDoS'd and I expect it to happen again. I know its done by some cronjobs but I need the actual commands to put there?

I also want to know which IP made how much input in mb/sec maximum as I have a high traffic so it would almost take me 6 hours to keep searching those files for the attacker's IP. So is there anything in WireShark during the analysis of those files which might tell how much input in mb/s was made by an IP to my server? If not, how should I find that?

Edit: *--------------------------------------------*

You guys are free to post your ideas of countering this as well. All I need is to find the attacker's IP, the packet-data he sent and the input in mb/s made to my server. My clients do not make more than 300kb/s input so if we set a filter to capture more than 1mb/s input if made, we could capture that.

share|improve this question
    
For a high traffic server, this sounds like suicide. –  Oliver May 11 '12 at 5:35
    
So in that case, could we just setup something that would log only the attack, lets say tcpdump capturing packets only greater than 1mb/s? –  Asad Moeen May 11 '12 at 5:55
    
You could for example monitor the number of packets on your network interface and activate tcpdump for say 1000 packets if the traffic exceeds a certain threshold. But logging all of your traffic is insane. –  Oliver May 11 '12 at 6:26
    
Yeah that would be even better. Could you show me the command to capture if lets say some IP inputs more than 1mb/s ? –  Asad Moeen May 11 '12 at 6:28
add comment

6 Answers

up vote 6 down vote accepted

Instead of logging all traffic, I would suggest the following: Monitor the number of packets sent to your server. If it exceeds a certain threshold, log a couple of 1000 packets, then wait for a longer time.

That packet trace should contain plenty of information which can be used for analysis. Also, it will not impose too much additional load on your server while everything is fine. You could use the following hacked together bash code as a starting point (could be started in screen, for example):

interface=eth0
dumpdir=/tmp/

while /bin/true; do
  pkt_old=`grep $interface: /proc/net/dev | cut -d :  -f2 | awk '{ print $2 }'`
  sleep 1
  pkt_new=`grep $interface: /proc/net/dev | cut -d :  -f2 | awk '{ print $2 }'`

  pkt=$(( $pkt_new - $pkt_old ))
  echo -ne "\r$pkt packets/s\033[0K"

  if [ $pkt -gt 5000 ]; then
    echo -e "\n`date` Under attack, dumping packets."
    tcpdump -n -s0 -c 2000 -w $dumpdir/dump.`date +"%Y%m%d-%H%M%S"`.cap
    echo "`date` Packets dumped, sleeping now."
    sleep 300
  fi
done

Feel free to adapt it to your needs.

share|improve this answer
    
Thanks, that's great. I should just keep it running in a screen or as a startup script right? Also, what is the threshold in the script. I see greater than 5000 but is that kbps ? –  Asad Moeen May 12 '12 at 9:34
    
So maybe could you tell me about that? –  Asad Moeen Jun 3 '12 at 15:17
    
No offence, but if you don't know what to do with what I gave you, you should definitely hire somebody to do that work for you. –  Oliver Jun 4 '12 at 7:18
    
I am doing this for the first time, I know how to run that but I just wanted to know where in the script do I define threshold? I see its 5000 but the unit? –  Asad Moeen Jun 5 '12 at 8:05
1  
Well with just one point here in Server Fault - I can neither upvote, or comment. I just wanted to say thank you to @Oliver above for his excellent bash script. I was experiencing a UDP flood attack on my server, and this script helped me to identify and solve it by 'tripping' a tcpdump during 'peak' events Thanks Oliver. –  Anthony Bouch Dec 10 '12 at 0:02
show 1 more comment

It's right there in the man pages, tcpdump has -G,

If specified, rotates the dump file specified with the -w option every
rotate_seconds seconds. Savefiles will have the name specified by -w
which should include a time format as defined by strftime(3). If no
time format is specified, each new file will overwrite the previous.

So, tcpdump -i eth0 -s 65535 -G 86400 -w /var/log/caps/%F.pcap will write to /var/log/caps/%F.pcap (where %F will be 2012-05-10, 2012-05-11, 2012-05-12, etc). Keep in mind it will rotate 24hrs from the time you start the cap, so it's not technically per-day unless you run it at midnight.

I'm not saying what you're planning on doing is a good idea, just that this is the solution you're asking for.

share|improve this answer
    
That looks nice but why isn't it a good idea? –  Asad Moeen May 10 '12 at 19:43
1  
An unfiltered dump like this during a DDoS is going to capture a lot of traffic. Your server is already likely becoming unresponsive due to the attack, trying to write all that traffic out to disk is just going to load the server even more, and cause it to become unresponsive sooner. If the attack truly is a DDoS, it could be coming from hundreds of hosts, but the tcpdump could be useful to simply get a handle for what is happening during the attack. From there you can analyse the dump and look into appropriate counter measures, like iptable's connlimit module –  brent May 10 '12 at 20:16
    
Well yeah when the server was under the attack, I couldn't even login to SSH so it was totally impossible for me to catch it right at the same time. All my applications almost crashed. So I thought this would be a way around this. Also, do you think we can also get input in mb/s made by an IP or otherwise add more options to tcpdump to catch only those IPs that make more than 1mb/s input. That would filter the traffic only to the attacker because my clients don't input more than 300kb/s. –  Asad Moeen May 11 '12 at 5:32
1  
This may be possible with tc, but I don't have specific experience. On FreeBSD it would be possible with a non-GENERIC kernel that has DUMMYNET built in, via ipfw and pipes. –  brent May 14 '12 at 14:25
add comment

You can certainly get that data from tcpdump, but it's not entirely straighforward.

First, tcpdump writes to a special file format which isn't a log file, so you would need either another instance of tcpdump or Wireshark to analyze the logfiles. But here's a basic suggestion:

  • write a script that kills any running tcpdump and starts a new one which writes to a log file with the day's date in its name
  • run that script from cron every midnight
  • have a cron entry that clears files older than 3 days in the directory in which you store the log files

Be warned that tcpdump gives a lot of output, so you'll need a fair amount of free disk space!

share|improve this answer
    
Well yeah but I need to know the exact commands, I already know how it works but need to get the script to do it. Also, I know I need WireShark for analyzing those dump files. So basically I need something to run "tcpdump -w Output.log" for 24 hours / 3 days. –  Asad Moeen May 10 '12 at 18:55
    
To help with your script I'd need to know what OS you're running. Some commands, like ps, are different between different OS'es, and you'd need it to be able to restart the tcpdump command to get it to write a new file. –  Jenny D May 10 '12 at 18:59
    
I had another thought. Are you running any kind of firewall? You can set ipfilter to log even successful connections, maybe that'd give you enough information for you to get the stats you want. And since it logs via syslog, it's easy to get the log rotation you want. –  Jenny D May 10 '12 at 19:01
    
I am not running any Firewall, just simple iptables rules to go for protection I need. I am running Debian 6 - 64bit. –  Asad Moeen May 10 '12 at 19:36
add comment

If you are on Linux you could use logrotate.

Something like

   /var/log/dump.pcap {
       rotate 3
       daily
       postrotate
           /usr/bin/killall tcpdump
           /usr/sbin/tcpdump options -w /var/log/dump.pcap
       endscript
   }

This logrotate configuration would go into e.g. /etc/logrotate.d/tcpdump.

You probably have a either a line in /etc/crontab or like me a script /etc/cron.daily/logrotate that calls logrotate.

Logrotate will when it processes this file rename /var/log/dump.pcap.1 to /var/log/dump.pcap.2 and /var/log/dump.pcap to /var/log/dump.pcap.1 and so on. Then when all those files are renamed and the oldest ones removed (in this example /var/log/dump.pcap.2 would be removed before renaming .1 to .2) it will execute the commands in postrotate. Unfortunately tcpdump does not survice a kill -HUP that is used on other deamons like httpd so this recipe kills it and then starts a new capture.

Note that the first day you may want to start the tcpdump manually.

This is untested but should do the trick.

share|improve this answer
    
I am running Debian 6 64-bit. Sorry but what does this script do? As I get it, it would run " tcpdump options -w /var/log/dump.pcap " daily for 3 days and then kill it. Am I right? If yes, where does it rename the files for each day? –  Asad Moeen May 10 '12 at 19:41
    
What's the odds of missing packets between the killall and reinitializing tcpdump? Any way to tell? –  Aaron Copley Oct 16 '12 at 18:38
add comment

Something like darkstat might be more useful to identify high traffic hosts, although it won't store the actual traffic (it does record port numbers though).

share|improve this answer
    
That looks good but is it Real-Time recorder? –  Asad Moeen May 10 '12 at 19:37
add comment

I've used tshark to do this but you need to be careful.

tshark -i eth0 -a duraition:86400 -b -w x.pcap

Or you can set complex output format options and redirect stdout. The problem is tshark never discards received packets so it eventually runs out of memory. shorter runs are better.

Another technique I like is to use iptables and ULOG. there are several ulog daemons around that can send things to ordinary log files. I've also used specter to convert ulog reports to messages.

share|improve this answer
    
Hi. Should this run on tcpdump? I don't have a GUI. –  Asad Moeen May 10 '12 at 19:38
1  
tshark is the command line version of wireshark. Its like tcpdump with more options. –  Julian May 10 '12 at 20:04
    
Ah okay. Nice thanks. –  Asad Moeen May 12 '12 at 9:33
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.