3

We currently have a dual-NIC Ubuntu server at our data centre acting as the gateway router between our public networks and our ISP. We have a /30 cross connect network on the ISP-facing NIC, and one IP from each of the three networks attached to our interior-facing NIC.

I would like to configure network traffic statistic generation and collection on this server, using Cisco's Netflow protocols. This will allow me to confirm our ISP's billing, as well as break-down data flow within our network.

What tools or packages would you recommend to passively capture traffic statistics and record them for later processing? Extra points if the Netflow collector has a MySQL data-store connector.

Improve this question

6 Answers 6

Reset to default
2

For generating a tool like nprobe or fprobe will work fine as others have mentioned.

For collecting I like nfdump/nfsen. It doesn't use mysql, but it is really easy to work with and get data out of it in a machine readable form.

You probably don't want the full netflow data in mysql, instead it usually makes more sense to run an aggregation query and load the summary into mysql. Having 10,000,000 records in mysql is not going to be fun to work with, but inserting a daily or hourly summary of (ip,total flows,total bytes,total packets) works a lot better.

Improve this answer
1
  • +1 for SQL summaries. I store (ip, flows, in bytes, outbytes) every five minutes and the database is reasonably usable. I tried dumping netflow data into a database, and found it storage-intensive and incredibly slow to access. For specific flow information, linear searches through netflow files turned out to be faster in all cases (of the scenarios we tried anyways).
    – David Mackintosh
    Feb 26, 2010 at 4:11
1

I know argus can read and process netflow data and it is quite good at collecting and processing network flow data by itself.

I've never used it to create netflow data as I usually just use it to collect and process the data natively, or use it to take a variety of flow types (tcpdump captures, netflows, etc) and use argus for the aggregation and summarization and analysis.

Improve this answer
1
  • I used argus to generate netflows for tapped/mirrored ports, flows were send to flow-tools for processing. It worked great, I vaguely remember some data missing from flows, but nothing I needed.
    – monomyth
    Feb 25, 2010 at 0:56
1

nprobe netflow generator

And I personally use flow-tools to store flows on disk, generate reports.

Regards K

Edit: here are many more tools for logging to mysql, charting, etc.

Improve this answer
1

I suggest you looking at argus, as chris says. From my experience it's the best behaving flow collector. But there are good alternatives like flowd and pfflowd that might work for you too. If you have any decent load (terrabytes per day) forget about storing your flows in any SQL database :) oh, and yes, flow-tools are great once you learn all the filtering magic and such.

Improve this answer
0

I recommend checking out Damien Miller's tools Softflowd and flowd which are a software based NetFlow exporter and collector respectively.

The source is available and even includes some examples for storing data in a SQL database (see flowinsert.pl under the tools dir).

These are great tools that a made to do specific tasks and can be customized as needed. I recommend them as a good place to start for NetFlow analysis.

Improve this answer
0

If you have high amount of traffic (approaching or more than 1Gbit) I would recommend to look at ipt-netflow (configurable and supporting NetFlow v6/v9/ipfix). Nprobe, recommended before, even though stated as 'GPL' and 'Open Source' is commercial (as in 'buyable'). Other solutions may not handle high packet rates.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .