Netflow Collector

Collect Netflow data from a Cisco Router with a Perl program. Current version 1.07.

Synopsis

Collects NetFlow export packets sent from a router, performs some basic aggregation, and writes the collected data to a file for further processing later. Currently understands NetFlow export format versions 1, 5, and 6.

Configuration file options

• Port number
  
• Specify the UDP port to listen on.
  
• Host host_group => filename
  
• For the named group of hosts, log to this file. %y%m%d%h is replaced by year, month, day of month, and hour respectively, in localtime. If the letters are uppsercased, it does the same for GMT/UTC.
  
• IP IP_address => host_group
  
• All packets sent from this address are considered to be part of the named host group.
  
• Local IP_address/network_bits
  
• Specify local networks, for which sources and destinations are recorded on an IP-by-IP address instead of being aggreggated.
  
• DefaultFragmentSize number
  
• Specify the size of the blocks IP addresses are aggregated into if they are not specified in a "Local" declaration.
  
• KnownPort number
  
• Only one port is logged in any TCP flow. Normally the lowest port is deemed to be the significant one (eg, for a flow between ports 80 and 3127, port 80 is logged.) If the higher port is listed as a "known port", it is used instead. Normally used for specifying a high port-numbered proxy server.
  
• DumpTime number
  
• The amount of time between writes to the log file. The greater the time, the more aggregation, but the less granular the data is. Can be set to zero for no dumping, but a SIGUSR1 will still force a dump.
  
• LogFile filename
  
• Write messages to this file with a timestamp instead of to stderr. Almost alwaysused when running as a daemon or service.
  
• WarnInhbit
  
• Do not log anything to "stderr". Does not prevent things from being written to a log file. Mainly used to avoid startup messages before LogFile takes effect for installations that require complete silence.
  

Log file format

Fixed width format with space separators. Numbers in brackets indicate character positions in the line for this field.

1. Unix timestamp at time of writing this line. (1-11)
   
2. Source IP address. (12-26)
   
3. Inbound Interface (the router's SNMP ID of that interface in hexadecimal, 0 for internal). (28-31)
   
4. Destination IP address. (33-47)
   
5. Next Hop IP address. Often 0.0.0.0 if the route only set the interface and not the next hop IP, as is often the case on Dialer interfaces. (49-63)
   
6. Outbound Interface (as per inbound interface). (65-69)
   
7. Protocol Number - 1 for ICMP, 6 for TCP, and 17 for UDP are the most common. (71-73)
   
8. Port designation - < if the listed port was the source, > if the listed port was the destination port. (75)
   
9. Port Number (77-81)
   
10. Source Autonmous System Number, zero if not availible. (83-87)
    
11. Destination Autonmous System Number, zero if not availible. (89-93)
    
12. Source Layer 2 encapsulation overhead per packet, zero if not availible. (95-96)
    
13. Destination Layer 2 encapsulation overhead per packet, zero if not availible. (98-99)
    
14. Layer 3 bytes in the flow. (101-108)
    
15. Packets in the flow. (110-117)

To calculate Layer 2 bytes, take the layer 3 bytes and add the number of packets multiplied by the layer 2 encapsulation overhead for that interface.

Run Time Options

• A SIGHUP (normally -1) sent to the process in POSIX will cause the collector to re-read its configuration file.
  
• A SIGUSR1 signal sent to the process in POSIX will force a dump. This is usually used with DumpTime 0 in the configuration file to have an external process control when dumps are done.
  

POSIX Installation

• Download the NetFlow Collector for POSIX systems such as Unix, Linux, Mac OS X, BeOS, etc.
  
• Install into /usr/local/netflow, or wherever you feel is appropriate.
  
• Usually started on system startup by some startup script, an example (netflow.sh) is provided.

Note that it doesn't hold the log file open, so you can safely move the log file out from underneath it, and it will just create a new one at the next dump interval.

Windows 32 Installation

• Download the NetFlow Collector for Win32 and unpack into C:\NetFlow (assumed below) or C:\Program Files\Iagu\NetFlow. Note the code is identical to the POSIX version, the difference is that the configuration file is has a three character extension and the example location for storing the collected data has changed.
  
• Download ActivePerl from http://aspn.activestate.com/ASPN/Downloads/ActivePerl/
  
• You probably want to run it as a service on NT. You can run your Perl scripts as Windows NT Services via a program called srvany.exe, which comes with the Windows NT Resource Kit. Once srvany.exe is installed, read the srvany.wri file which should be with it. This document will explain how to set up registry entries for your new service. 
  

The command to run is:
srvany perl c:\netflow\netflow-collector.pl c:\netflow\netflow-collector.ini

Contact info@iagu.net for further information.