Just moozing

Before you can check your notes, you must make them…

PCAP analysis and Tstat

with 3 comments

As part of our monitoring solution, we needed to generate RRD graphs from off-line PCAPS. I found Tstat which turned out to be a really good solution.

We have had a lot of problem with our off-line analysis of network traffic. There are just so much data – we had 500+ GB – and most of the software is designed to do on-line real time processing. Tstat had the option of crunching the numbers off-line, which was what we needed.

 

Installation

The installation has been done on Debian, but it is the same or very similar on Ubuntu.

Install base packages


apt-get install libpcap-dev libpcap0.8 automake librrd-dev libtool build-essential rrdtool

 

Download and build tstat. On a fast machine this takes < 5 min.


mkdir tstat
cd tstat
wget http://tstat.polito.it/download/tstat-2.4.tar.gz
tar xvf tstat-2.4.tar.gz
cd tstat-2.4/
./autogen.sh
./configure --enable-libtstat
make

The binary is locate at ./tstat/tstat.

 

Using Tstat

Just running Tstat will get you started


tstat

 

Adding RRD


tstat -r RRD -R rrd.conf <somefile.pcap>

ip_bitrate_loc_w_prot

You can get a sample rrd.conf from the source file at ./tstat-conf/rrd.conf. The resulting RRD databases are found in the ./RRD directory.

In my first attempts, I though it didn’t work since the graphs showed nothing. It turned to be a scaling issues – I had processed < 2 minutes of traffic, and it was invisible on the graphs.

RRD is a cool format that is worth looking into more. It is a fixed size database manipulated using rrdtools and with lots of interesting bindings for, say, python. I generate graphs using something like the following:


#!/bin/sh

ENDTIME="-0m"
DATA="ip_bitrate_loc"
RRDDIR="RRD"

rrdtool graph \
  ${DATA}_w_prot.png \
  --lazy --start end-2h --end $ENDTIME \
  --title "dataset: $DATA" \
  --vertical-label "bits/sec" \
  DEF:tcp=$RRDDIR/$DATA.idx0.rrd:$DATA:AVERAGE \
  DEF:udp=$RRDDIR/$DATA.idx1.rrd:$DATA:AVERAGE \
  DEF:icmp=$RRDDIR/$DATA.idx2.rrd:$DATA:AVERAGE \
  LINE1:tcp#ff0000:"TCP " \
  GPRINT:tcp:AVERAGE:"Avg\: %3.2lf %sbps\t" \
  GPRINT:tcp:MAX:"Max\: %3.2lf %sbps\t" \
  GPRINT:tcp:MIN:"Min\: %3.2lf %sbps\l" \
  LINE2:udp#00ff00:"UDP " \
  GPRINT:udp:AVERAGE:"Avg\: %3.2lf %sbps\t" \
  GPRINT:udp:MAX:"Max\: %3.2lf %sbps\t" \
  GPRINT:udp:MIN:"Min\: %3.2lf %sbps\l" \
  LINE3:icmp#0000ff:ICMP \
  GPRINT:icmp:AVERAGE:"Avg\: %3.2lf %sbps\t" \
  GPRINT:icmp:MAX:"Max\: %3.2lf %sbps\t" \
  GPRINT:icmp:MIN:"Min\: %3.2lf %sbps\l"

We combine tstat with mergecap as I described in a earlier post.

Advertisements

Written by moozing

March 27, 2015 at 08:00

Posted in Tech

Tagged with , , ,

3 Responses

Subscribe to comments with RSS.

  1. There were some issues compiling it on SecurityOnion (which would be running on Ubuntu 12.04 – 3.13.0-45-generic #74~precise1-Ubuntu).

    Issue 1
    ./autogen.sh: 6: ./autogen.sh: autoreconf: not found
    Fixed by installing the dh-autoreconf package

    Issue 2
    configure: error: missing ‘pcap’ library
    Fixed by installing the libpcap-dev package

    Victor

    March 27, 2015 at 10:20

    • Issue 3
      rrd support
      Fixed by installing librrd-dev

      Victor

      March 27, 2015 at 15:28

      • I tried to rebuild it in a vnilla ubuntu 12.04, and follwing my instructions in the post, I had no problems compiling it. Perhaps there is some minor difference between sec onion and the vanilla ubuntu.

        I did get segfaults, but that was due to broken pcap data (verified using tshark and editcap)

        moozing

        April 1, 2015 at 21:10


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: