.\" Automatically generated by Pod::Man 4.11 (Pod::Simple 3.35)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "FLOWDUMPER 1p"
.TH FLOWDUMPER 1p "2019-10-06" "perl v5.30.0" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
flowdumper \- a grep(1)\-like utility for raw flow files
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
.Vb 1
\&   flowdumper [\-h] [\-v] [\-s|S|r|R] [\-a|n] [[\-I expr] \-e expr [\-E expr]] [\-c] [\-B file] [\-o output_file] [flow_file [...]]
.Ve
.PP
but usually just:
.PP
.Vb 1
\&   flowdumper [\-s] \-e expr flow_file [...]
.Ve
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
\&\fBflowdumper\fR is a \fBgrep\fR\|(1)\-like utility for selecting and processing
flows from cflowd or flow-tools raw flow files.  The selection criteria
are specified by using the \f(CW\*(C`\-e\*(C'\fR option described below.
.PP
\&\fBflowdumper\fR's primary features are the ability to:
.IP "\(bu" 4
Print the content of raw flow files in one of two built-in formats or a
format of the users own.  The built-in \*(L"long\*(R" format is much like that
produced by the flowdump command supplied with cflowd.  The \*(L"short\*(R",
single-line format is suitable for subsequent post-processing by
line-oriented filters like \fBsed\fR\|(1).
.IP "\(bu" 4
Act as a filter, reading raw flow input from either file(s) or standard
input, and producing filtered raw flow output on standard output.  This
is similar to how \fBgrep\fR\|(1) is often used on text files.
.IP "\(bu" 4
Select flows according to practically any criteria that can be
expressed in perl syntax.
.PP
The \*(L"flow variables\*(R" and other symbols available for use in the \f(CW\*(C`\-e\*(C'\fR
expression are those made available by the Cflow module when used like
this:
.PP
.Vb 1
\&   use Cflow qw(:flowvars :tcpflags :icmptypes :icmpcodes);
.Ve
.PP
See the Cflow perl documentation for full details on these values (i.e.
\&\*(L"perldoc Cflow\*(R".)
.PP
Most perl syntax is allowed in the expressions specified with the \f(CW\*(C`\-e\*(C'\fR,
\&\f(CW\*(C`\-I\*(C'\fR, and \f(CW\*(C`\-E\*(C'\fR options.  See the perl man pages for full details on
operators (\*(L"man perlop\*(R") and functions (\*(L"man perlfunc\*(R") available for
use in those expressions.
.PP
If run with no arguments, filters standard input to standard output.
.PP
The options and their arguments, roughly in order of usefulness, are:
.ie n .IP """\-h""" 4
.el .IP "\f(CW\-h\fR" 4
.IX Item "-h"
shows the usage information
.Sp
mnemonic: 'h'elp
.ie n .IP """\-a""" 4
.el .IP "\f(CW\-a\fR" 4
.IX Item "-a"
print all flows
.Sp
implied if \f(CW\*(C`\-e\*(C'\fR is not specified
.Sp
mnemonic: 'a'll
.ie n .IP """\-e"" expr" 4
.el .IP "\f(CW\-e\fR expr" 4
.IX Item "-e expr"
evaluate this expression once per flow
.Sp
mnemonic: 'e'xpression
.ie n .IP """\-c""" 4
.el .IP "\f(CW\-c\fR" 4
.IX Item "-c"
print number of flows matched in input
.Sp
mnemonic: 'c'ount
.ie n .IP """\-s""" 4
.el .IP "\f(CW\-s\fR" 4
.IX Item "-s"
print flows in short (one-line) format, ignored with \f(CW\*(C`\-n\*(C'\fR
.Sp
mnemonic: 's'hort
.ie n .IP """\-r""" 4
.el .IP "\f(CW\-r\fR" 4
.IX Item "-r"
print flows in the raw/binary flow file format
.Sp
ignored with \f(CW\*(C`\-n\*(C'\fR
.Sp
mnemonic: 'r'aw
.ie n .IP """\-R""" 4
.el .IP "\f(CW\-R\fR" 4
.IX Item "-R"
\&\*(L"repacks\*(R" and print flows in the raw/binary flow file format
.Sp
requires \f(CW\*(C`\-e\*(C'\fR, ignored with \f(CW\*(C`\-n\*(C'\fR, useful with \f(CW\*(C`\-p\*(C'\fR
.Sp
mnemonic: 'R'epack raw
.ie n .IP """\-n""" 4
.el .IP "\f(CW\-n\fR" 4
.IX Item "-n"
don't print matching flows
.Sp
mnemonic: like "perl \f(CW\*(C`\-n\*(C'\fR\*(L" or \*(R"sed \f(CW\*(C`\-n\*(C'\fR"
.ie n .IP """\-o"" output_file" 4
.el .IP "\f(CW\-o\fR output_file" 4
.IX Item "-o output_file"
send output to the specified file.  A single \fBprintf\fR\|(3) string
conversion specifier can be used within the output_file value (such as
\&\*(L"/tmp/%s.txt\*(R") to make the output file name a function of the input
file basename.
.Sp
mneomic: 'o'utput file
.ie n .IP """\-S""" 4
.el .IP "\f(CW\-S\fR" 4
.IX Item "-S"
print flows in the \*(L"old\*(R" short (one-line) format
.Sp
ignored with \f(CW\*(C`\-n\*(C'\fR
.Sp
mnemonic: 'S'hort
.ie n .IP """\-v""" 4
.el .IP "\f(CW\-v\fR" 4
.IX Item "-v"
be verbose with messages
.Sp
mnemonic: 'v'erbose
.ie n .IP """\-V""" 4
.el .IP "\f(CW\-V\fR" 4
.IX Item "-V"
be very verbose with messages (implies "\f(CW\*(C`\-v\*(C'\fR")
.Sp
mnemonic: 'V'ery verbose
.ie n .IP """\-I"" expr" 4
.el .IP "\f(CW\-I\fR expr" 4
.IX Item "-I expr"
eval expression initially, before flow processing
.Sp
practically useless without \f(CW\*(C`\-e\*(C'\fR
.Sp
mnemonic: 'I'nitial expression
.ie n .IP """\-E"" expr" 4
.el .IP "\f(CW\-E\fR expr" 4
.IX Item "-E expr"
eval expression after flow processing is complete
.Sp
practically useless without \f(CW\*(C`\-e\*(C'\fR
.Sp
mnemonic: 'E'\s-1ND\s0 expression
.ie n .IP """\-B"" file" 4
.el .IP "\f(CW\-B\fR file" 4
.IX Item "-B file"
Load the specified \s-1BGP\s0 dump file using Net::ParseRouteTable.
.Sp
In your optional expression, you can now refer to these variables:
.Sp
.Vb 6
\&   $dst_as_path_arrayref
\&   $dst_origin_as
\&   $dst_peer_as
\&   $src_as_path_arrayref
\&   $src_origin_as
\&   $src_peer_as
.Ve
.Sp
which will cause a lookup.  Their values are undefined if the lookup fails.
.Sp
mnemonic: 'B'\s-1GP\s0 dump file
.ie n .IP """\-p"" prefix_mappings_file" 4
.el .IP "\f(CW\-p\fR prefix_mappings_file" 4
.IX Item "-p prefix_mappings_file"
read file containing IPv4 prefix mappings in this format (one per line):
.Sp
.Vb 2
\&   10.42.69.0/24 \-> 10.69.42.0/24
\&   ...
.Ve
.Sp
When specifying this option, you can, and should at some point, call
the \s-1ENCODE\s0 subroutine in your expressions to have it encode the \s-1IP\s0
address flowvars such as \f(CW$Cflow::exporter\fR, \f(CW$Cflow::srcaddr\fR,
\&\f(CW$Cflow::dstaddr\fR, and \f(CW$Cflow::nexthop\fR.
.Sp
mnemonic: 'p'refixes
.SH "EXAMPLES"
.IX Header "EXAMPLES"
Print all flows, in a multi-line format, to a pager:
.PP
.Vb 1
\&   $ flowdumper \-a flows.* |less
.Ve
.PP
Print all the \s-1UDP\s0 flows to another file using the raw binary flow format:
.PP
.Vb 1
\&   $ flowdumper \-re \*(Aq17 == $protocol\*(Aq flows.current > udp_flows.current
.Ve
.PP
Print all \s-1TCP\s0 flows which have the \s-1SYN\s0 bit set in the \s-1TCP\s0 flags:
.PP
.Vb 1
\&   $ flowdumper \-se \*(Aq6 == $protocol && ($TH_SYN & $tcp_flags)\*(Aq flows.*
.Ve
.PP
Print the first 10 flows to another file using the raw binary flow format:
.PP
.Vb 1
\&   $ flowdumper \-I \*(Aq$n = 10\*(Aq \-re \*(Aq$n\-\- or exit\*(Aq flows.*0 > head.cflow
.Ve
.PP
Print all flows with the start and end time using a two-line format:
.PP
.Vb 1
\&   $ flowdumper \-se \*(Aqprint scalar(localtime($startime)), "\en"\*(Aq flows.*
.Ve
.PP
Print all flows with the specified source address using a short,
single-line format:
.PP
.Vb 1
\&   $ flowdumper \-se \*(Aq"10.42.42.42" eq $srcip\*(Aq flows.*
.Ve
.PP
Do the same thing in a quicker, but less obvious, way:
.PP
.Vb 4
\&   $ flowdumper \-I \*(Aq
\&      use Socket;
\&      $addr = unpack("N", Socket::inet_aton("10.42.42.42"));
\&   \*(Aq \-se \*(Aq$addr == $srcaddr\*(Aq  flows.*
.Ve
.PP
(This latter method runs quicker because \fBinet_aton\fR\|(3) is only called
once, instead of once per flow.)
.PP
Print all flows with a source address within the specifed network/subnet:
.PP
.Vb 5
\&   $ flowdumper \e
\&   \-I \*(Aquse Socket;
\&       $mask = unpack("N", Socket::inet_aton("10.42.0.0"));
\&       $width = 16\*(Aq \e
\&   \-se \*(Aq$mask == ((0xffffffff << (32\-$width)) & $srcaddr)\*(Aq flows.*
.Ve
.PP
Print all flows where either the source or the destination address, but
not both, is within the specified set of networks or subnets:
.PP
.Vb 7
\&   $ flowdumper \e
\&   \-I \*(Aquse Net::Patricia;
\&       $pt = Net::Patricia\->new;
\&       map { $pt\->add_string($_, 1) } qw( 10.42.0.0/16
\&                                          10.69.0.0/16 )\*(Aq \e
\&   \-se \*(Aq1 == ($pt\->match_integer($srcaddr) +
\&              $pt\->match_integer($dstaddr))\*(Aq flows.*
.Ve
.PP
Count the total number of \*(L"talkers\*(R" (unique source host addresses) by
piping them to \fBsort\fR\|(1) and \fBwc\fR\|(1) to count them:
.PP
.Vb 7
\&   $ flowdumper \e
\&   \-I \*(Aquse Net::Patricia;
\&       $pt = Net::Patricia\->new;
\&       map { $pt\->add_string($_, 1) } qw( 10.42.0.0/16
\&                                          10.69.0.0/16 )\*(Aq \e
\&   \-ne \*(Aq$pt\->match_integer($srcaddr) and print "$srcip\en"\*(Aq flows.* \e
\&   |sort \-u |wc \-l
.Ve
.PP
Count the total number of \*(L"talkers\*(R" (unique source host addresses) that
are within a the specified networks or subnets:
.PP
.Vb 10
\&   $ flowdumper \e
\&   \-I \*(Aquse Net::Patricia;
\&       $pt = new Net::Patricia;
\&       map { $pt\->add_string($_, 1) } qw( 10.42.0.0/16
\&                                          10.69.0.0/16 );
\&       $talkers = new Net::Patricia\*(Aq \e
\&   \-ne \*(Aq$pt\->match_integer($srcaddr) &&
\&        ($talkers\->match_integer($srcaddr) or
\&         $talkers\->add_string($srcip, 1))\*(Aq \e
\&   \-E \*(Aqprintf("%d\en", $talkers\->climb( sub { 1 } ))\*(Aq flows.*
.Ve
.PP
(For large numbers of flows, this latter method is quicker because it
populates a Net::Patricia trie with the unique addresses and counts the
resulting nodes rather than having to print them to standard output and
then having to sort them to determine how many are unique.)
.PP
Select the \s-1TCP\s0 flows and \*(L"\s-1ENCODE\*(R"\s0 the \s-1IP\s0 addresses according to the
prefix encodings specified in \*(L"prefix_encodings.txt\*(R":
.PP
.Vb 1
\&   $ flowdumper \-p prefix_encodings.txt \-se \*(Aq6 == $protocol && ENCODE\*(Aq
.Ve
.PP
Produce a new raw flow file with the \s-1IP\s0 addresses ENCODEd according to
the prefix encodings specified in \*(L"prefix_encodings.txt\*(R":
.PP
.Vb 1
\&   $ flowdumper \-p prefix_encodings.txt \-Re \*(AqENCODE\*(Aq flows > flows.enc
.Ve
.PP
Produce a set of raw flow files that have the \f(CW$src_as\fR and \f(CW$dst_as\fR
origin \s-1AS\s0 values filled in based upon a lookup in externally-specified
routing table (in the file \*(L"router.bgp\*(R") and have the \s-1IP\s0 address info
replaces with zeroes (for anonymity):
.PP
.Vb 1
\&   $ ssh router "show route protocol bgp terse" > router.bgp # Juniper
\&
\&   $ flowdumper \e
\&   \-B router.bgp \e
\&   \-e \*(Aq$src_as = $src_origin_as,
\&       $dst_as = $dst_origin_as,
\&       (($exporter = 0),
\&        ($srcaddr  = 0),
\&        ($src_mask = 0),
\&        ($dstaddr  = 0),
\&        ($dst_mask = 0),
\&        ($nexthop  = 0), 1)\*(Aq \e
\&   \-R \e
\&   \-o /tmp/%s.cflow_enc \e
\&   flows*
.Ve
.SH "NOTES"
.IX Header "NOTES"
This utility was inspired by Daniel McRobb's \fBflowdump\fR utility which
is supplied with cflowd.  \fBflowdumper\fR was originally written as
merely a sample of what can be done with the Cflow perl module, but has
since been developed into a more complete tool.
.SH "BUGS"
.IX Header "BUGS"
When using the \f(CW\*(C`\-B\*(C'\fR option, routing table entries that contain \s-1AS\s0 sets
at the end of the \s-1AS\s0 path are quietly discarded.  (It's not so quiet if
you also specified \f(CW\*(C`\-V\*(C'\fR.)  It was necessary to discard these, because I
did not consider \s-1AS\s0 sets when designing the \s-1API\s0 and therefore have no
way to communicate more than one origin \s-1AS\s0 value per for a single
source or destination \s-1IP\s0 address.
.PP
There are perhaps some pathological combinations of options that
currently do not produce usage error messages, but should.
.PP
Since the expression syntax is that of perl itself, there are lots of
useless expressions that will happily be accepted without complaint.
This is particular troublesome when trying to track down typos, for
instance, with the flow variable names.
.PP
This script probably has the same bugs as the Cflow module, since it's
based upon it.
.SH "AUTHOR"
.IX Header "AUTHOR"
Dave Plonka <plonka@doit.wisc.edu>
.PP
Copyright (C) 1998\-2002  Dave Plonka.  This program is free
software; you can redistribute it and/or modify it under the
terms of the \s-1GNU\s0 General Public License as published by the
Free Software Foundation; either version 2 of the License,
or (at your option) any later version.
.SH "SEE ALSO"
.IX Header "SEE ALSO"
\&\fBperl\fR\|(1), Socket, Net::Netmask, Net::Patricia, Cflow.