5

What I basically want is to to write all tcpdump captured packets to a file every 3 days. So basically tcpdump should run for 24 hours on day 1 and write the output to Day1.log and similar for Day2 and Day3. On the 4th day it should repeat and write the log to Day1 again. This is basically to check DDoS attempts on my server and to find out the type of attack including the attacker's IP as in the last 7 days my machines were DDoS'd and I expect it to happen again. I know its done by some cronjobs but I need the actual commands to put there?

I also want to know which IP made how much input in mb/sec maximum as I have a high traffic so it would almost take me 6 hours to keep searching those files for the attacker's IP. So is there anything in WireShark during the analysis of those files which might tell how much input in mb/s was made by an IP to my server? If not, how should I find that?

Edit: --------------------------------------------

You guys are free to post your ideas of countering this as well. All I need is to find the attacker's IP, the packet-data he sent and the input in mb/s made to my server. My clients do not make more than 300kb/s input so if we set a filter to capture more than 1mb/s input if made, we could capture that.

4
  • For a high traffic server, this sounds like suicide.
    – Oliver
    May 11, 2012 at 5:35
  • So in that case, could we just setup something that would log only the attack, lets say tcpdump capturing packets only greater than 1mb/s?
    – Asad Moeen
    May 11, 2012 at 5:55
  • You could for example monitor the number of packets on your network interface and activate tcpdump for say 1000 packets if the traffic exceeds a certain threshold. But logging all of your traffic is insane.
    – Oliver
    May 11, 2012 at 6:26
  • Yeah that would be even better. Could you show me the command to capture if lets say some IP inputs more than 1mb/s ?
    – Asad Moeen
    May 11, 2012 at 6:28

6 Answers 6

7

Instead of logging all traffic, I would suggest the following: Monitor the number of packets sent to your server. If it exceeds a certain threshold, log a couple of 1000 packets, then wait for a longer time.

That packet trace should contain plenty of information which can be used for analysis. Also, it will not impose too much additional load on your server while everything is fine. You could use the following hacked together bash code as a starting point (could be started in screen, for example):

interface=eth0
dumpdir=/tmp/

while /bin/true; do
  pkt_old=`grep $interface: /proc/net/dev | cut -d :  -f2 | awk '{ print $2 }'`
  sleep 1
  pkt_new=`grep $interface: /proc/net/dev | cut -d :  -f2 | awk '{ print $2 }'`

  pkt=$(( $pkt_new - $pkt_old ))
  echo -ne "\r$pkt packets/s\033[0K"

  if [ $pkt -gt 5000 ]; then
    echo -e "\n`date` Under attack, dumping packets."
    tcpdump -n -s0 -c 2000 -w $dumpdir/dump.`date +"%Y%m%d-%H%M%S"`.cap
    echo "`date` Packets dumped, sleeping now."
    sleep 300
  fi
done

Feel free to adapt it to your needs.

6
  • Thanks, that's great. I should just keep it running in a screen or as a startup script right? Also, what is the threshold in the script. I see greater than 5000 but is that kbps ?
    – Asad Moeen
    May 12, 2012 at 9:34
  • So maybe could you tell me about that?
    – Asad Moeen
    Jun 3, 2012 at 15:17
  • No offence, but if you don't know what to do with what I gave you, you should definitely hire somebody to do that work for you.
    – Oliver
    Jun 4, 2012 at 7:18
  • I am doing this for the first time, I know how to run that but I just wanted to know where in the script do I define threshold? I see its 5000 but the unit?
    – Asad Moeen
    Jun 5, 2012 at 8:05
  • 1
    Well with just one point here in Server Fault - I can neither upvote, or comment. I just wanted to say thank you to @Oliver above for his excellent bash script. I was experiencing a UDP flood attack on my server, and this script helped me to identify and solve it by 'tripping' a tcpdump during 'peak' events Thanks Oliver. Dec 10, 2012 at 0:02
15

It's right there in the man pages, tcpdump has -G,

If specified, rotates the dump file specified with the -w option every
rotate_seconds seconds. Savefiles will have the name specified by -w
which should include a time format as defined by strftime(3). If no
time format is specified, each new file will overwrite the previous.

So, tcpdump -i eth0 -s 65535 -G 86400 -w /var/log/caps/%F.pcap will write to /var/log/caps/%F.pcap (where %F will be 2012-05-10, 2012-05-11, 2012-05-12, etc). Keep in mind it will rotate 24hrs from the time you start the cap, so it's not technically per-day unless you run it at midnight.

I'm not saying what you're planning on doing is a good idea, just that this is the solution you're asking for.

4
  • That looks nice but why isn't it a good idea?
    – Asad Moeen
    May 10, 2012 at 19:43
  • 2
    An unfiltered dump like this during a DDoS is going to capture a lot of traffic. Your server is already likely becoming unresponsive due to the attack, trying to write all that traffic out to disk is just going to load the server even more, and cause it to become unresponsive sooner. If the attack truly is a DDoS, it could be coming from hundreds of hosts, but the tcpdump could be useful to simply get a handle for what is happening during the attack. From there you can analyse the dump and look into appropriate counter measures, like iptable's connlimit module
    – brent
    May 10, 2012 at 20:16
  • Well yeah when the server was under the attack, I couldn't even login to SSH so it was totally impossible for me to catch it right at the same time. All my applications almost crashed. So I thought this would be a way around this. Also, do you think we can also get input in mb/s made by an IP or otherwise add more options to tcpdump to catch only those IPs that make more than 1mb/s input. That would filter the traffic only to the attacker because my clients don't input more than 300kb/s.
    – Asad Moeen
    May 11, 2012 at 5:32
  • 1
    This may be possible with tc, but I don't have specific experience. On FreeBSD it would be possible with a non-GENERIC kernel that has DUMMYNET built in, via ipfw and pipes.
    – brent
    May 14, 2012 at 14:25
4

You can certainly get that data from tcpdump, but it's not entirely straighforward.

First, tcpdump writes to a special file format which isn't a log file, so you would need either another instance of tcpdump or Wireshark to analyze the logfiles. But here's a basic suggestion:

  • write a script that kills any running tcpdump and starts a new one which writes to a log file with the day's date in its name
  • run that script from cron every midnight
  • have a cron entry that clears files older than 3 days in the directory in which you store the log files

Be warned that tcpdump gives a lot of output, so you'll need a fair amount of free disk space!

4
  • Well yeah but I need to know the exact commands, I already know how it works but need to get the script to do it. Also, I know I need WireShark for analyzing those dump files. So basically I need something to run "tcpdump -w Output.log" for 24 hours / 3 days.
    – Asad Moeen
    May 10, 2012 at 18:55
  • To help with your script I'd need to know what OS you're running. Some commands, like ps, are different between different OS'es, and you'd need it to be able to restart the tcpdump command to get it to write a new file.
    – Jenny D
    May 10, 2012 at 18:59
  • I had another thought. Are you running any kind of firewall? You can set ipfilter to log even successful connections, maybe that'd give you enough information for you to get the stats you want. And since it logs via syslog, it's easy to get the log rotation you want.
    – Jenny D
    May 10, 2012 at 19:01
  • I am not running any Firewall, just simple iptables rules to go for protection I need. I am running Debian 6 - 64bit.
    – Asad Moeen
    May 10, 2012 at 19:36
3

If you are on Linux you could use logrotate.

Something like

   /var/log/dump.pcap {
       rotate 3
       daily
       postrotate
           /usr/bin/killall tcpdump
           /usr/sbin/tcpdump options -w /var/log/dump.pcap
       endscript
   }

This logrotate configuration would go into e.g. /etc/logrotate.d/tcpdump.

You probably have a either a line in /etc/crontab or like me a script /etc/cron.daily/logrotate that calls logrotate.

Logrotate will when it processes this file rename /var/log/dump.pcap.1 to /var/log/dump.pcap.2 and /var/log/dump.pcap to /var/log/dump.pcap.1 and so on. Then when all those files are renamed and the oldest ones removed (in this example /var/log/dump.pcap.2 would be removed before renaming .1 to .2) it will execute the commands in postrotate. Unfortunately tcpdump does not survice a kill -HUP that is used on other deamons like httpd so this recipe kills it and then starts a new capture.

Note that the first day you may want to start the tcpdump manually.

This is untested but should do the trick.

2
  • I am running Debian 6 64-bit. Sorry but what does this script do? As I get it, it would run " tcpdump options -w /var/log/dump.pcap " daily for 3 days and then kill it. Am I right? If yes, where does it rename the files for each day?
    – Asad Moeen
    May 10, 2012 at 19:41
  • What's the odds of missing packets between the killall and reinitializing tcpdump? Any way to tell? Oct 16, 2012 at 18:38
1

Something like darkstat might be more useful to identify high traffic hosts, although it won't store the actual traffic (it does record port numbers though).

1
  • That looks good but is it Real-Time recorder?
    – Asad Moeen
    May 10, 2012 at 19:37
0

I've used tshark to do this but you need to be careful.

tshark -i eth0 -a duraition:86400 -b -w x.pcap

Or you can set complex output format options and redirect stdout. The problem is tshark never discards received packets so it eventually runs out of memory. shorter runs are better.

Another technique I like is to use iptables and ULOG. there are several ulog daemons around that can send things to ordinary log files. I've also used specter to convert ulog reports to messages.

2
  • Hi. Should this run on tcpdump? I don't have a GUI.
    – Asad Moeen
    May 10, 2012 at 19:38
  • 1
    tshark is the command line version of wireshark. Its like tcpdump with more options.
    – Julian
    May 10, 2012 at 20:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .