.\" Man page generated from reStructuredText.
.
.TH "PYTHON-IPFIX" "1" "Mar 07, 2020" "0.9" "python-ipfix"
.SH NAME
python-ipfix \- python-ipfix Documentation
.
.nr rst2man-indent-level 0
.
.de1 rstReportMargin
\\$1 \\n[an-margin]
level \\n[rst2man-indent-level]
level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
-
\\n[rst2man-indent0]
\\n[rst2man-indent1]
\\n[rst2man-indent2]
..
.de1 INDENT
.\" .rstReportMargin pre:
. RS \\$1
. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
. nr rst2man-indent-level +1
.\" .rstReportMargin post:
..
.de UNINDENT
. RE
.\" indent \\n[an-margin]
.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
.nr rst2man-indent-level -1
.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
..
.sp
IPFIX implementation for Python 3.3.
.sp
This module provides a Python interface to IPFIX message streams, and
provides tools for building IPFIX Exporting and Collecting Processes.
It handles message framing and deframing, encoding and decoding IPFIX
data records using templates, and a bridge between IPFIX ADTs and
appropriate Python data types.
.sp
Before using any of the functions of this module, it is necessary to populate
the information model with Information Elements.
\fI\%ipfix.ie.use_iana_default()\fP populates the default IANA IPFIX Information
Element Registry shipped with the module; this is the current registry as of
release time. \fI\%ipfix.ie.use_5103_default()\fP populates the reverse
counterpart IEs as in \fI\%RFC 5103\fP\&. The module also supports the definition of 
enterprise\-specific Information Elements via \fI\%ipfix.ie.for_spec()\fP and 
\fI\%ipfix.ie.use_specfile()\fP; see \fI\%ipfix.ie\fP for more.
.sp
For reading and writing of records to IPFIX message streams with automatic
message boundary management, see the \fI\%ipfix.reader\fP and
\fI\%ipfix.writer\fP modules, respectively. For manual reading and writing of
messages, see \fI\%ipfix.message\fP\&. In any case, exporters will need to define
templates; see \fI\%ipfix.template\fP\&.
.sp
This module is copyright 2013 Brian Trammell. It is made available under the
terms of the 
\fI\%GNU Lesser General Public License\fP, 
version 3 or, at your option, any later version.
.sp
Reference documentation for each module is found in the subsections below.
.SH MODULE IPFIX.TYPES
.sp
Implementation of IPFIX abstract data types (ADT) and mappings to Python types.
.sp
Maps each IPFIX ADT to the corresponding Python type, as below:
.TS
center;
|l|l|.
_
T{
IPFIX Type
T}	T{
Python Type
T}
_
T{
octetArray
T}	T{
bytes
T}
_
T{
unsigned8
T}	T{
int
T}
_
T{
unsigned16
T}	T{
int
T}
_
T{
unsigned32
T}	T{
int
T}
_
T{
unsigned64
T}	T{
int
T}
_
T{
signed8
T}	T{
int
T}
_
T{
signed16
T}	T{
int
T}
_
T{
signed32
T}	T{
int
T}
_
T{
signed64
T}	T{
int
T}
_
T{
float32
T}	T{
float
T}
_
T{
float64
T}	T{
float
T}
_
T{
boolean
T}	T{
bool
T}
_
T{
macAddress
T}	T{
bytes
T}
_
T{
string
T}	T{
str
T}
_
T{
dateTimeSeconds
T}	T{
datetime
T}
_
T{
dateTimeMilliseconds
T}	T{
datetime
T}
_
T{
dateTimeMicroseconds
T}	T{
datetime
T}
_
T{
dateTimeNanoseconds
T}	T{
datetime
T}
_
T{
ipv4Address
T}	T{
ipaddress
T}
_
T{
ipv6Address
T}	T{
ipaddress
T}
_
.TE
.sp
Though client code generally will not use this module directly, it defines how
each IPFIX abstract data type will be represented in Python, and the concrete
IPFIX representation of each type. Type methods operate on buffers, as used
internally by the \fI\%ipfix.message.MessageBuffer\fP class, so we\(aqll create 
one to illustrate encoding and decoding:
.sp
.nf
.ft C
>>> import ipfix.types
>>> buf = memoryview(bytearray(16))
.ft P
.fi
.sp
Each of the encoding methods returns the offset into the buffer of the first
byte after the encoded value; since we\(aqre always encoding to the beginning
of the buffer in this example, this is equivalent to the length. 
We use this to bound the encoded value on subsequent decode.
.sp
Integers are represented by the python int type:
.sp
.nf
.ft C
>>> unsigned32 = ipfix.types.for_name("unsigned32")
>>> length = unsigned32.encode_single_value_to(42, buf, 0)
>>> buf[0:length].tolist()
[0, 0, 0, 42]
>>> unsigned32.decode_single_value_from(buf, 0, length)
42
.ft P
.fi
.sp
\&...floats by the float type, with the usual caveats about precision:
.sp
.nf
.ft C
>>> float32 = ipfix.types.for_name("float32")
>>> length = float32.encode_single_value_to(42.03579, buf, 0)
>>> buf[0:length].tolist()
[66, 40, 36, 166]
>>> float32.decode_single_value_from(buf, 0, length)
42.035789489746094
.ft P
.fi
.sp
\&...strings by the str type, encoded as UTF\-8:
.sp
.nf
.ft C
>>> string = ipfix.types.for_name("string")
>>> length = string.encode_single_value_to("Grüezi", buf, 0)
>>> buf[0:length].tolist()
[71, 114, 195, 188, 101, 122, 105]
>>> string.decode_single_value_from(buf, 0, length)
\(aqGrüezi\(aq
.ft P
.fi
.sp
\&...addresses as the IPv4Address and IPv6Address types in the ipaddress module:
.sp
.nf
.ft C
>>> from ipaddress import ip_address
>>> ipv4Address = ipfix.types.for_name("ipv4Address")
>>> length = ipv4Address.encode_single_value_to(ip_address("198.51.100.27"), buf, 0)
>>> buf[0:length].tolist()
[198, 51, 100, 27]
>>> ipv4Address.decode_single_value_from(buf, 0, length)
IPv4Address(\(aq198.51.100.27\(aq)
>>> ipv6Address = ipfix.types.for_name("ipv6Address")
>>> length = ipv6Address.encode_single_value_to(ip_address("2001:db8::c0:ffee"), buf, 0)
>>> buf[0:length].tolist()
[32, 1, 13, 184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 192, 255, 238]
>>> ipv6Address.decode_single_value_from(buf, 0, length)
IPv6Address(\(aq2001:db8::c0:ffee\(aq)
.ft P
.fi
.sp
\&...and the timestamps of various precision as a python datetime, 
encoded as per RFC5101bis:
.sp
.nf
.ft C
>>> from datetime import datetime
>>> from datetime import timezone
>>> dtfmt = "%Y\-%m\-%d %H:%M:%S.%f"
>>> dt = datetime.strptime("2013\-06\-21 14:00:03.456789", dtfmt)
.ft P
.fi
.sp
dateTimeSeconds truncates microseconds:
.sp
.nf
.ft C
>>> dateTimeSeconds = ipfix.types.for_name("dateTimeSeconds")
>>> length = dateTimeSeconds.encode_single_value_to(dt, buf, 0)
>>> buf[0:length].tolist()
[81, 196, 92, 99]
>>> dateTimeSeconds.decode_single_value_from(buf, 0, length).strftime(dtfmt)
\(aq2013\-06\-21 14:00:03.000000\(aq
.ft P
.fi
.sp
dateTimeMilliseconds truncates microseconds to the nearest millisecond:
.sp
.nf
.ft C
>>> dateTimeMilliseconds = ipfix.types.for_name("dateTimeMilliseconds")
>>> length = dateTimeMilliseconds.encode_single_value_to(dt, buf, 0)
>>> buf[0:length].tolist()
[0, 0, 1, 63, 103, 8, 228, 128]
>>> dateTimeMilliseconds.decode_single_value_from(buf, 0, length).strftime(dtfmt)
\(aq2013\-06\-21 14:00:03.456000\(aq
.ft P
.fi
.sp
dateTimeMicroseconds exports microseconds fully in NTP format:
.sp
.nf
.ft C
>>> dateTimeMicroseconds = ipfix.types.for_name("dateTimeMicroseconds")
>>> length = dateTimeMicroseconds.encode_single_value_to(dt, buf, 0)
>>> buf[0:length].tolist()
[81, 196, 92, 99, 116, 240, 32, 0]
>>> dateTimeMicroseconds.decode_single_value_from(buf, 0, length).strftime(dtfmt)
\(aq2013\-06\-21 14:00:03.456789\(aq
.ft P
.fi
.sp
dateTimeNanoseconds is also supported, but is identical to
dateTimeMicroseconds, as the datetime class in Python only supports
microsecond\-level timing.
.INDENT 0.0
.TP
.B class ipfix.types.IpfixType(name, num, valenc, valdec, valstr, valparse, roottype=None)
Abstract interface for all IPFIX types. Used internally.
.UNINDENT
.INDENT 0.0
.TP
.B exception ipfix.types.IpfixTypeError(*args)
Raised when attempting to do an unsupported operation on a type
.UNINDENT
.INDENT 0.0
.TP
.B class ipfix.types.OctetArrayType(name, num, valenc=<function _identity>, valdec=<function _identity>, valstr=<built\-in function hexlify>, valparse=<built\-in function unhexlify>, roottype=None)
Type encoded by byte array packing. Used internally.
.UNINDENT
.INDENT 0.0
.TP
.B class ipfix.types.StructType(name, num, stel, valenc=<function _identity>, valdec=<function _identity>, valstr=<class \(aqstr\(aq>, valparse=<class \(aqint\(aq>, roottype=None)
Type encoded by struct packing. Used internally.
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.types.decode_varlen(buf, offset)
Decode a IPFIX varlen encoded length; used internally by template
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.types.encode_varlen(buf, offset, length)
Encode a IPFIX varlen encoded length; used internally by template
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.types.for_name(name)
Return an IPFIX type for a given type name
.INDENT 7.0
.TP
.B Parameters
\fBname\fP \-\- the name of the type to look up
.TP
.B Returns
IpfixType \-\- type instance for that name
.TP
.B Raises
IpfixTypeError
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.types.use_integer_ipv4()
Use integers instead of ipaddress.IPv4Address to store IPv4 addresses.
Changes behavior globally; should be called before using any IPFIX types.
Designed for use with numpy arrays, to not require a Python object for
storing IP addresses.
.UNINDENT
.SH MODULE IPFIX.IE
.sp
IESpec\-based interface to IPFIX information elements,
and interface to use the default IPFIX IANA Information Model
.sp
An IESpec is a string representation of an IPFIX information element,
including all the information required to define it, as documented in
Section 9 of \fI\%http://tools.ietf.org/html/draft\-ietf\-ipfix\-ie\-doctors\fP\&.
It has the format:
.INDENT 0.0
.INDENT 3.5
name(pen/num)<type>[size]
.UNINDENT
.UNINDENT
.sp
To specify a new Information Element, a complete IESpec
must be passed to for_spec():
.sp
.nf
.ft C
>>> import ipfix.ie
>>> e = ipfix.ie.for_spec("myNewInformationElement(35566/1)<string>")
>>> e
InformationElement(\(aqmyNewInformationElement\(aq, 35566, 1, ipfix.types.for_name(\(aqstring\(aq), 65535)
.ft P
.fi
.sp
The string representation of an InformationElement is its IESpec:
.sp
.nf
.ft C
>>> str(e)
\(aqmyNewInformationElement(35566/1)<string>[65535]\(aq
.ft P
.fi
.sp
To get an Information Element already specified, an incomplete specification
can be passed; a name or number is enough:
.sp
.nf
.ft C
>>> ipfix.ie.use_iana_default()
>>> ipfix.ie.use_5103_default()
>>> str(ipfix.ie.for_spec("octetDeltaCount"))
\(aqoctetDeltaCount(0/1)<unsigned64>[8]\(aq
>>> str(ipfix.ie.for_spec("(2)"))
\(aqpacketDeltaCount(0/2)<unsigned64>[8]\(aq
.ft P
.fi
.sp
Reduced\-length encoding and fixed\-length sequence types are supported by the
for_length method; this is used internally by templates.
.sp
.nf
.ft C
>>> str(e.for_length(32))
\(aqmyNewInformationElement(35566/1)<string>[32]\(aq
.ft P
.fi
.sp
An Information Element object can also be used to translate 
between native Python and string representations of an Information Element value:
.sp
.nf
.ft C
>>> ipfix.ie.for_spec("sourceIPv4Address").parse("192.0.2.19")
IPv4Address(\(aq192.0.2.19\(aq)
>>> from datetime import datetime
>>> ipfix.ie.for_spec("flowEndMilliseconds").unparse(datetime(2013,6,21,14))
\(aq2013\-06\-21 14:00:00.000\(aq
.ft P
.fi
.sp
Most client code will only need the \fI\%use_iana_default()\fP, 
\fI\%use_5103_default()\fP, and \fI\%use_specfile()\fP functions; 
client code using tuple interfaces will need \fI\%spec_list()\fP as well.
.INDENT 0.0
.TP
.B class ipfix.ie.InformationElement(name, pen, num, ietype=ipfix.types.for_name(\(aqoctetArray\(aq), length=None, valstr=None, valparse=None)
An IPFIX Information Element (IE). This is essentially a five\-tuple of
name, element number (num), a private enterprise number (pen; 0 if it
is an IANA registered IE), a type, and a length.
.sp
Information Elements may also have value string and parser functions,
for representing the values as strings; if not set, these default to
.sp
InformationElement instances should be obtained using the \fI\%for_spec()\fP
or \fI\%for_template_entry()\fP functions.
.INDENT 7.0
.TP
.B for_length(length)
Get an instance of this IE for the specified length.
Used to support reduced\-length encoding (RLE).
.INDENT 7.0
.TP
.B Parameters
\fBlength\fP \-\- length of the new IE
.TP
.B Returns
this IE if length matches, or a new IE for the length
.TP
.B Raises
ValueError
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B parse(s)
Parse a string to a value using the conversion function 
for this Information Element. Uses the default string conversion
for the IE\(aqs type if not overridden at IE creation time.
.INDENT 7.0
.TP
.B Parameters
\fBs\fP \-\- string to parse using this IEs\(aqs string conversion
.TP
.B Returns
value for given string
.TP
.B Raises
ValueError is not a valid string representation for this IE
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B unparse(v)
Unparse a value to a string using the conversion function 
for this Information Element. Uses the default string conversion
for the IE\(aqs type if not overridden at IE creation time.
.INDENT 7.0
.TP
.B Parameters
\fBv\fP \-\- value to unparse using this IEs\(aqs string conversion
.TP
.B Returns
string representation of v
.TP
.B Raises
ValueError if v is not a valid value for this IE
.UNINDENT
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B class ipfix.ie.InformationElementList(iterable=None)
A hashable ordered list of Information Elements.
.sp
Used internally by templates, and to specify the
order of tuples to the tuple append and iterator interfaces. Get an
instance by calling \fI\%spec_list()\fP
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.clear_infomodel()
Reset the cache of known Information Elements.
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.for_spec(spec)
Get an IE from the cache of known IEs, or create a
new IE if not found, given an IESpec.
.INDENT 7.0
.TP
.B Parameters
\fBspec\fP \-\- IEspec, as in draft\-ietf\-ipfix\-ie\-doctors, of the 
form name(pen/num)<type>[size]; some fields may be
omitted unless creating a new IE in the cache.
.TP
.B Returns
an IE for the name
.TP
.B Raises
ValueError
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.for_template_entry(pen, num, length)
Get an IE from the cache of known IEs, or create a
new IE if not found, given a private enterprise number,
element number, and length. Used internally by Templates.
.INDENT 7.0
.TP
.B Parameters
.INDENT 7.0
.IP \(bu 2
\fBpen\fP \-\- private enterprise number, or 0 for an IANA IE
.IP \(bu 2
\fBnum\fP \-\- IE number (Element ID)
.IP \(bu 2
\fBlength\fP \-\- length of the IE in bytes
.UNINDENT
.TP
.B Returns
an IE for the given pen, num, and length. If the IE has not
been previously added to the cache of known IEs, the IE will be 
named _ipfix_pen_num, and have octetArray as a type.
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.parse_spec(spec)
Parse an IESpec into name, pen, number, typename, and length fields
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.spec_list(specs)
Given a list or iterable of IESpecs, return a hashable list of IEs.
Pass this as the ielist argument to the tuple export
and iterator functions.
.INDENT 7.0
.TP
.B Parameters
\fBspecs\fP \-\- list of IESpecs
.TP
.B Returns
a new Information Element List, suitable for use with the tuple
export and iterator functions in \fBmessage\fP
.TP
.B Raises
ValueError
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.use_5103_default()
Load the module internal list of RFC 5103 reverse IEs for IANA
registered IEs into the cache of known IEs. Normally, biflow\-aware
client code should call this just after use_iana_default().
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.use_iana_default()
Load the module internal list of IANA registered IEs into the
cache of known IEs. Normally, client code should call this before
using any other part of this module.
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.ie.use_specfile(filename)
Load a file listing IESpecs into the cache of known IEs
.INDENT 7.0
.TP
.B Parameters
\fBfilename\fP \-\- name of file containing IESpecs to open
.TP
.B Raises
ValueError
.UNINDENT
.UNINDENT
.SH MODULE IPFIX.TEMPLATE
.sp
Representation of IPFIX templates.
Provides template\-based packing and unpacking of data in IPFIX messages.
.sp
For reading, templates are handled internally. For writing, use 
\fBfrom_ielist()\fP to create a template.
.sp
See \fI\%ipfix.message\fP for examples.
.INDENT 0.0
.TP
.B exception ipfix.template.IpfixDecodeError(*args)
Raised when decoding a malformed IPFIX message
.UNINDENT
.INDENT 0.0
.TP
.B exception ipfix.template.IpfixEncodeError(*args)
Raised on internal encoding errors, or if message MTU is too small
.UNINDENT
.INDENT 0.0
.TP
.B class ipfix.template.Template(tid=0, iterable=None)
An IPFIX Template.
.sp
A template is an ordered list of IPFIX Information Elements with an ID.
.INDENT 7.0
.TP
.B append(ie)
Append an IE to this Template
.UNINDENT
.INDENT 7.0
.TP
.B count()
Count IEs in this template
.UNINDENT
.INDENT 7.0
.TP
.B decode_from(buf, offset, packplan=None)
Decodes a record into a tuple containing values in template order
.UNINDENT
.INDENT 7.0
.TP
.B decode_namedict_from(buf, offset, recinf=None)
Decodes a record from a buffer into a dict keyed by IE name.
.UNINDENT
.INDENT 7.0
.TP
.B decode_tuple_from(buf, offset, recinf=None)
Decodes a record from a buffer into a tuple,
ordered as the IEs in the InformationElementList given as recinf.
.UNINDENT
.INDENT 7.0
.TP
.B encode_namedict_to(buf, offset, rec, recinf=None)
Encodes a record from a dict containing values keyed by IE name
.UNINDENT
.INDENT 7.0
.TP
.B encode_template_to(buf, offset, setid)
Encodes the template to a buffer.
Encodes as a Template if setid is TEMPLATE_SET_ID,
as an Options Template if setid is OPTIONS_SET_ID.
.UNINDENT
.INDENT 7.0
.TP
.B encode_to(buf, offset, vals, packplan=None)
Encodes a record from a tuple containing values in template order
.UNINDENT
.INDENT 7.0
.TP
.B encode_tuple_to(buf, offset, rec, recinf=None)
Encodes a record from a tuple containing values ordered as the IEs 
in the template.
.UNINDENT
.INDENT 7.0
.TP
.B finalize()
Compile a default packing plan. Called after append()ing all IEs.
.UNINDENT
.INDENT 7.0
.TP
.B fixlen_count()
Count of fixed\-length IEs in this template before the first
variable\-length IE; this is the size of the portion of the template
which can be encoded/decoded efficiently.
.UNINDENT
.INDENT 7.0
.TP
.B identical_to(other)
Determine if two templates are identical to each other.
.sp
Two templates are considered identical if they contain the same 
IEs in the same order, and the same scope count. Template ID
is not considered as part of the test for template identity.
.UNINDENT
.INDENT 7.0
.TP
.B packplan_for_ielist
Given a list of IEs, devise and cache a packing plan.
Used by the tuple interfaces.
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B class ipfix.template.TemplatePackingPlan(tmpl, indices)
Plan to pack/unpack a specific set of indices for a template.
Used internally by Templates for efficient encoding and decoding.
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.template.decode_template_from(buf, offset, setid)
Decodes a template from a buffer.
Decodes as a Template if setid is TEMPLATE_SET_ID,
as an Options Template if setid is OPTIONS_SET_ID.
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.template.for_specs(tid, *specs)
Create a template from a template ID and a list of IESpecs
.INDENT 7.0
.TP
.B Parameters
.INDENT 7.0
.IP \(bu 2
\fBtid\fP \-\- Template ID, must be between 256 and 65535.
.IP \(bu 2
\fB*specs\fP \-\- 
.sp
List of IESpecs

.UNINDENT
.TP
.B Returns
A new Template, ready to use for writing to a Message
.UNINDENT
.UNINDENT
.SH MODULE IPFIX.MESSAGE
.sp
Provides the MessageBuffer class for encoding and decoding IPFIX Messages.
.sp
This interface allows direct control over Messages; for reading or writing
records automatically from/to streams, see \fI\%ipfix.reader\fP and
\fI\%ipfix.writer\fP, respectively.
.sp
To create a message buffer:
.sp
.nf
.ft C
>>> import ipfix.message
>>> msg = ipfix.message.MessageBuffer()
>>> msg
<MessageBuffer domain 0 length 0>
.ft P
.fi
.sp
To prepare the buffer to write records:
.sp
.nf
.ft C
>>> msg.begin_export(8304)
>>> msg
<MessageBuffer domain 8304 length 16 (writing)>
.ft P
.fi
.sp
Note that the buffer grows to contain the message header.
.sp
To write records to the buffer, first you\(aqll need a template:
.sp
.nf
.ft C
>>> import ipfix.ie
>>> ipfix.ie.use_iana_default()
>>> import ipfix.template
>>> tmpl = ipfix.template.from_ielist(256, 
\&...        ipfix.ie.spec_list(("flowStartMilliseconds",
\&...                            "sourceIPv4Address",
\&...                            "destinationIPv4Address",
\&...                            "packetDeltaCount")))
>>> tmpl
<Template ID 256 count 4 scope 0>
.ft P
.fi
.sp
To add the template to the message:
.sp
.nf
.ft C
>>> msg.add_template(tmpl)
>>> msg
<MessageBuffer domain 8304 length 40 (writing set 2)>
.ft P
.fi
.sp
Note that \fI\%MessageBuffer.add_template()\fP exports the template when it 
is written by default, and that the current set ID is 2 (template set).
.sp
Now, a set must be created to add records to the message; the set ID must match
the ID of the template. MessageBuffer automatically uses the template matching
the set ID for record encoding.
.sp
.nf
.ft C
>>> msg.export_ensure_set(256)
>>> msg
<MessageBuffer domain 8304 length 44 (writing set 256)>
.ft P
.fi
.sp
Records can be added to the set either as dictionaries keyed by IE name:
.sp
.nf
.ft C
>>> from datetime import datetime
>>> from ipaddress import ip_address
>>> rec = { "flowStartMilliseconds" : datetime.strptime("2013\-06\-21 14:00:00", 
\&...                                       "%Y\-%m\-%d %H:%M:%S"),
\&...         "sourceIPv4Address" : ip_address("10.1.2.3"),
\&...         "destinationIPv4Address" : ip_address("10.5.6.7"),
\&...         "packetDeltaCount" : 27 }
>>> msg.export_namedict(rec)
>>> msg
<MessageBuffer domain 8304 length 68 (writing set 256)>
.ft P
.fi
.sp
or as tuples in template order:
.sp
.nf
.ft C
>>> rec = (datetime.strptime("2013\-06\-21 14:00:02", "%Y\-%m\-%d %H:%M:%S"),
\&...        ip_address("10.8.9.11"), ip_address("10.12.13.14"), 33)
>>> msg.export_tuple(rec)
>>> msg
<MessageBuffer domain 8304 length 92 (writing set 256)>
.ft P
.fi
.sp
Variable\-length information elements will be encoded using the native length
of the passed value:
.sp
.nf
.ft C
>>> ipfix.ie.for_spec("myNewInformationElement(35566/1)<string>")
InformationElement(\(aqmyNewInformationElement\(aq, 35566, 1, ipfix.types.for_name(\(aqstring\(aq), 65535)
>>> tmpl = ipfix.template.from_ielist(257, 
\&...        ipfix.ie.spec_list(("flowStartMilliseconds",
\&...                            "myNewInformationElement")))
>>> msg.add_template(tmpl)
>>> msg.export_ensure_set(257)
>>> msg
<MessageBuffer domain 8304 length 116 (writing set 257)>
>>> rec = { "flowStartMilliseconds" : datetime.strptime("2013\-06\-21 14:00:04", 
\&...                                   "%Y\-%m\-%d %H:%M:%S"),
\&...         "myNewInformationElement" : "Grüezi, Y\(aqall" }
>>> msg.export_namedict(rec)
>>> msg
<MessageBuffer domain 8304 length 139 (writing set 257)>
.ft P
.fi
.sp
Attempts to write past the end of the message (set via the mtu parameter, 
default 65535) result in \fI\%EndOfMessage\fP being raised.
.sp
Messages can be written to a stream using \fI\%MessageBuffer.write_message()\fP, 
or dumped to a byte array for transmission using \fI\%MessageBuffer.to_bytes()\fP\&.
The message must be reset before starting to write again.
.sp
.nf
.ft C
>>> b = msg.to_bytes()
>>> msg.begin_export()
>>> msg 
<MessageBuffer domain 8304 length 16 (writing)>
.ft P
.fi
.sp
Reading happens more or less in reverse. To begin, a message is read from a
byte array using \fI\%MessageBuffer.from_bytes()\fP, or from a stream using 
\fI\%MessageBuffer.read_message()\fP\&.
.sp
.nf
.ft C
>>> msg.from_bytes(b)
>>> msg
<MessageBuffer domain 8304 length 139 (deframed 4 sets)>
.ft P
.fi
.sp
Both of these methods scan the message in advance to find the sets within
the message. The records within these sets can then be accessed by iterating
over the message. As with export, the records can be accessed as a dictionary 
mapping IE names to values or as tuples. The dictionary interface is
designed for general IPFIX processing applications, such as collectors 
accepting many types of data, or diagnostic tools for debugging IPFIX export:
.sp
.nf
.ft C
>>> for rec in msg.namedict_iterator():
\&...    print(sorted(rec.items()))
\&...
[(\(aqdestinationIPv4Address\(aq, IPv4Address(\(aq10.5.6.7\(aq)), (\(aqflowStartMilliseconds\(aq, datetime.datetime(2013, 6, 21, 14, 0)), (\(aqpacketDeltaCount\(aq, 27), (\(aqsourceIPv4Address\(aq, IPv4Address(\(aq10.1.2.3\(aq))]
[(\(aqdestinationIPv4Address\(aq, IPv4Address(\(aq10.12.13.14\(aq)), (\(aqflowStartMilliseconds\(aq, datetime.datetime(2013, 6, 21, 14, 0, 2)), (\(aqpacketDeltaCount\(aq, 33), (\(aqsourceIPv4Address\(aq, IPv4Address(\(aq10.8.9.11\(aq))]
[(\(aqflowStartMilliseconds\(aq, datetime.datetime(2013, 6, 21, 14, 0, 4)), (\(aqmyNewInformationElement\(aq, "Grüezi, Y\(aqall")]
.ft P
.fi
.sp
The tuple interface for reading messages is designed for applications with a
specific internal data model. It can be much faster than the dictionary
interface, as it skips decoding of IEs not requested by the caller, and can
skip entire sets not containing all the requested IEs. Requested IEs are
specified as an \fI\%ipfix.ie.InformationElementList\fP instance, from 
\fBie.spec_list()\fP:
.sp
.nf
.ft C
>>> ielist = ipfix.ie.spec_list(["flowStartMilliseconds", "packetDeltaCount"])
>>> for rec in msg.tuple_iterator(ielist):
\&...     print(rec)
\&...
(datetime.datetime(2013, 6, 21, 14, 0), 27)
(datetime.datetime(2013, 6, 21, 14, 0, 2), 33)
.ft P
.fi
.sp
Notice that the variable\-length record written to the message are not returned 
by this iterator, since that record doesn\(aqt include a packetDeltaCount IE. 
The record is, however, still there:
.sp
.nf
.ft C
>>> ielist = ipfix.ie.spec_list(["myNewInformationElement"])
>>> for rec in msg.tuple_iterator(ielist):
\&...     print(rec)
\&...
("Grüezi, Y\(aqall",)
.ft P
.fi
.INDENT 0.0
.TP
.B exception ipfix.message.EndOfMessage(*args)
Exception raised when a write operation on a Message
fails because there is not enough space in the message.
.UNINDENT
.INDENT 0.0
.TP
.B class ipfix.message.MessageBuffer
Implements a buffer for reading or writing IPFIX messages.
.INDENT 7.0
.TP
.B active_template_ids()
Get an iterator over all active template IDs in the current domain.
Provided to allow callers to export some or all active Templates across
multiple Messages.
.INDENT 7.0
.TP
.B Returns
a template ID iterator
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B add_template(tmpl, export=True)
Add a template to this MessageBuffer. Adding a template makes it 
available for use for exporting records; see \fI\%export_new_set()\fP\&.
.INDENT 7.0
.TP
.B Parameters
.INDENT 7.0
.IP \(bu 2
\fBtmpl\fP \-\- the template to add
.IP \(bu 2
\fBexport\fP \-\- If True, export this template to the MessageBuffer
after adding it.
.UNINDENT
.TP
.B Raises
EndOfMessage
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B begin_export(odid=None)
Start exporting a new message. Clears any previous message content,
but keeps template information intact. Sets the message sequence number.
.INDENT 7.0
.TP
.B Parameters
\fBodid\fP \-\- Observation domain ID to use for export. By default, uses
the observation domain ID of the previous message. Note
that templates are scoped to observation domain, so
templates will need to be added after switching to a new
observation domain ID.
.TP
.B Raises
IpfixEncodeError
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B delete_template(tid, export=True)
Delete a template by ID from this MessageBuffer.
.INDENT 7.0
.TP
.B Parameters
.INDENT 7.0
.IP \(bu 2
\fBtid\fP \-\- ID of the template to delete
.IP \(bu 2
\fBexport\fP \-\- if True, export a Template Withdrawal for this
Template after deleting it
.UNINDENT
.TP
.B Raises
EndOfMessage
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B export_ensure_set(setid)
Ensure that the current set for export has the given Set ID.
Starts a new set if not using \fI\%export_new_set()\fP
.INDENT 7.0
.TP
.B Parameters
\fBsetid\fP \-\- Set ID of the new Set; corresponds to the Template ID of
the Template that will be used to encode records into the
Set. The require Template must have already been added
to the MessageBuffer, see \fI\%add_template()\fP\&.
.TP
.B Raises
IpfixEncodeError, EndOfMessage
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B export_namedict(rec)
Export a record to the message, using the template for the current Set
ID. The record is a dictionary mapping IE names to values. The
dictionary must contain a value for each IE in the template. Keys in the
dictionary not in the template will be ignored.
.INDENT 7.0
.TP
.B Parameters
\fBrec\fP \-\- the record to export, as a dictionary
.TP
.B Raises
EndOfMessage
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B export_needs_flush()
True if content has been written to this MessageBuffer since the
last call to \fI\%begin_export()\fP
.UNINDENT
.INDENT 7.0
.TP
.B export_new_set(setid)
Start exporting a new Set with the given set ID. Creates a new set
even if the current Set has the given set ID; client code should in most
cases use \fI\%export_ensure_set()\fP instead.
.INDENT 7.0
.TP
.B Parameters
\fBsetid\fP \-\- Set ID of the new Set; corresponds to the Template ID of
the Template that will be used to encode records into the
Set. The require Template must have already been added
to the MessageBuffer, see \fI\%add_template()\fP\&.
.TP
.B Raises
IpfixEncodeError, EndOfMessage
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B export_record(rec, encode_fn=<function Template.encode_namedict_to>, recinf=None)
Low\-level interface to record export.
.sp
Export a record to a MessageBuffer, using the template associated with
the Set ID given to the most recent \fI\%export_new_set()\fP or
\fI\%export_ensure_set()\fP call, and the given encode function. By
default, the record is assumed to be a dictionary mapping IE names
to values (i.e., the same as \fI\%export_namedict()\fP).
.INDENT 7.0
.TP
.B Parameters
.INDENT 7.0
.IP \(bu 2
\fBencode_fn\fP \-\- Function used to encode a record; 
must be an (unbound) "encode" instance method of the 
\fI\%ipfix.template.Template\fP class.
.IP \(bu 2
\fBrecinf\fP \-\- Record information opaquely passed to decode function
.UNINDENT
.TP
.B Raises
EndOfMessage
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B export_template(tid)
Export a template to this Message given its template ID.
.INDENT 7.0
.TP
.B Parameters
\fBtid\fP \-\- ID of template to export; must have been added to this
message previously with \fI\%add_template()\fP\&.
.TP
.B Raises
EndOfMessage, KeyError
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B export_tuple(rec)
Export a record to the message, using the template for the current Set
ID. The record is a tuple of values in template order.
.INDENT 7.0
.TP
.B Parameters
\fBrec\fP \-\- the record to export, as a tuple in template order.
.TP
.B Raises
EndOfMessage
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B from_bytes(bytes)
Read an IPFIX message from a byte array.
.sp
This populates message header fields and the internal setlist.
Call for each new message before iterating over records when reading
from a byte array.
.INDENT 7.0
.TP
.B Parameters
\fBbytes\fP \-\- a byte array containing a complete IPFIX message.
.TP
.B Raises
IpfixDecodeError
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B get_export_time()
Return the export time of this message. When reading, returns the 
export time as read from the message header. When writing, this is 
the argument of the last call to \fI\%set_export_time()\fP, or, if 
:attr:auto_export_time is True, the time of the last message
export.
.INDENT 7.0
.TP
.B Returns
export time of the last message read/written.
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B namedict_iterator()
Iterate over all records in the Message, as dicts mapping IE names
to values.
.INDENT 7.0
.TP
.B Returns
a name dictionary iterator
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B read_message(stream)
Read a IPFIX message from a stream.
.sp
This populates message header fields and the internal setlist.
Call for each new message before iterating over records when reading
from a stream.
.INDENT 7.0
.TP
.B Parameters
\fBstream\fP \-\- stream to read from
.TP
.B Raises
IpfixDecodeError
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B record_iterator(decode_fn=<function Template.decode_namedict_from>, tmplaccept_fn=<function accept_all_templates>, recinf=None)
Low\-level interface to record iteration.
.sp
Iterate over records in an IPFIX message previously read with 
\fI\%read_message()\fP or \fI\%from_bytes()\fP\&. Automatically handles 
templates in set order. By default, iterates over each record in the 
stream as a dictionary mapping IE name to value 
(i.e., the same as \fI\%namedict_iterator()\fP)
.INDENT 7.0
.TP
.B Parameters
.INDENT 7.0
.IP \(bu 2
\fBdecode_fn\fP \-\- Function used to decode a record; 
must be an (unbound) "decode" instance method of the 
\fI\%ipfix.template.Template\fP class.
.IP \(bu 2
\fBtmplaccept_fn\fP \-\- Function returning True if the given template
is of interest to the caller, False if not.
Default accepts all templates. Sets described by
templates for which this function returns False
will be skipped.
.IP \(bu 2
\fBrecinf\fP \-\- Record information opaquely passed to decode function
.UNINDENT
.TP
.B Returns
an iterator over records decoded by decode_fn.
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B set_export_time(dt=None)
Set the export time for the next message written with 
\fI\%write_message()\fP or \fI\%to_bytes()\fP\&. Disables automatic export 
time updates. By default, sets the export time to the current time.
.INDENT 7.0
.TP
.B Parameters
\fBdt\fP \-\- export time to set, as a datetime
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B template_for_id(tid)
Retrieve a Template for a given ID in the current domain.
.INDENT 7.0
.TP
.B Parameters
\fBtid\fP \-\- template ID to get
.TP
.B Returns
the template
.TP
.B Raises
KeyError
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B to_bytes()
Convert this MessageBuffer to a byte array, suitable for writing
to a binary file, socket, or datagram. Finalizes the message by
rewriting the message header with current length, and export time.
.INDENT 7.0
.TP
.B Returns
message as a byte array
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B tuple_iterator(ielist)
Iterate over all records in the Message containing all the IEs in 
the given ielist. Records are returned as tuples in ielist order.
.INDENT 7.0
.TP
.B Parameters
\fBielist\fP \-\- an instance of \fI\%ipfix.ie.InformationElementList\fP
listing IEs to return as a tuple
.TP
.B Returns
a tuple iterator for tuples as in ielist order
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B write_message(stream)
Convenience method to write a message to a stream; see \fI\%to_bytes()\fP\&.
.UNINDENT
.UNINDENT
.SH MODULE IPFIX.READER
.sp
Interface to read IPFIX Messages from a stream.
.INDENT 0.0
.TP
.B class ipfix.reader.MessageStreamReader(stream)
Reads records from a stream of IPFIX messages.
.sp
Uses an \fI\%ipfix.message.MessageBuffer\fP internally, and continually reads
messages from the given stream into the buffer, iterating over records,
until the end of the stream. Use \fI\%from_stream()\fP to get an instance.
.sp
Suitable for reading from IPFIX files (see \fI\%RFC 5655\fP) as well as from
UDP or TCP sockets with \fBsocketserver.StreamRequestHandler\fP\&. 
When opening a stream from a file, use mode=\(aqrb\(aq.
.INDENT 7.0
.TP
.B namedict_iterator()
Iterate over all records in the stream, as dicts mapping IE names
to values.
.INDENT 7.0
.TP
.B Returns
a name dictionary iterator
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B tuple_iterator(ielist)
Iterate over all records in the stream containing all the IEs in 
the given ielist. Records are returned as tuples in ielist order.
.INDENT 7.0
.TP
.B Parameters
\fBielist\fP \-\- an instance of \fI\%ipfix.ie.InformationElementList\fP
listing IEs to return as a tuple
.TP
.B Returns
a tuple iterator for tuples in ielist order
.UNINDENT
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.reader.from_stream(stream)
Get a MessageStreamReader for a given stream
.INDENT 7.0
.TP
.B Parameters
\fBstream\fP \-\- stream to read
.TP
.B Returns
a \fI\%MessageStreamReader\fP wrapped around the stream.
.UNINDENT
.UNINDENT
.SH MODULE IPFIX.WRITER
.INDENT 0.0
.TP
.B class ipfix.writer.MessageStreamWriter(stream, mtu=65535)
Writes records to a stream of IPFIX messages.
.sp
Uses an \fI\%ipfix.message.MessageBuffer\fP internally, and continually
writes records into messages, exporting messages to the stream each time the 
maximum message size (MTU) is reached. Use \fI\%to_stream()\fP to get an
instance.
.sp
Suitable for writing to IPFIX files (see \fI\%RFC 5655\fP) as well as to TCP 
sockets. When writing a stream to a file, use mode=\(aqwb\(aq.
.INDENT 7.0
.TP
.B \&..warning: This class is not yet suitable for UDP export; this is an open
issue to be fixed in a subsequent release.
.UNINDENT
.INDENT 7.0
.TP
.B add_template(tmpl)
Add a template to this Writer. Adding a template makes it 
available for use for exporting records; see \fI\%set_export_template()\fP\&.
.INDENT 7.0
.TP
.B Parameters
\fBtmpl\fP \-\- the template to add
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B export_namedict(rec)
Export a record to the message, using the current template
The record is a dictionary mapping IE names to values. The
dictionary must contain a value for each IE in the template. Keys in the
dictionary not in the template will be ignored.
.INDENT 7.0
.TP
.B Parameters
\fBrec\fP \-\- the record to export, as a dictionary
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B flush()
Export an in\-progress Message immediately.
.sp
Used internally to manage message boundaries, but
can also be used to force immediate export (e.g. to reduce delay
due to buffer dwell time), as well as to finish write operations on
a Writer before closing the underlying stream.
.UNINDENT
.INDENT 7.0
.TP
.B set_domain(odid)
Sets the observation domain for subsequent messages sent with 
this Writer.
.INDENT 7.0
.TP
.B Parameters
\fBodid\fP \-\- Observation domain ID to use for export. Note that
templates are scoped to observation domain, so 
templates will need to be added after switching to a 
new observation domain ID.
.UNINDENT
.UNINDENT
.INDENT 7.0
.TP
.B set_export_template(tid)
Set the template to be used for export by subsequent calls to
\fI\%export_namedict()\fP and \fBexport_tuple()\fP\&.
.INDENT 7.0
.TP
.B Parameters
\fBtid\fP \-\- Template ID of the Template that will be used to encode 
records to the Writer. The corresponding Template must 
have already been added to the Writer, see 
\fI\%add_template()\fP\&.
.UNINDENT
.UNINDENT
.UNINDENT
.INDENT 0.0
.TP
.B ipfix.writer.to_stream(stream, mtu=65535)
Get a MessageStreamWriter for a given stream
.INDENT 7.0
.TP
.B Parameters
.INDENT 7.0
.IP \(bu 2
\fBstream\fP \-\- stream to write
.IP \(bu 2
\fBmtu\fP \-\- maximum message size in bytes; defaults to 65535,
the largest possible ipfix message.
.UNINDENT
.TP
.B Returns
a \fI\%MessageStreamWriter\fP wrapped around the stream.
.UNINDENT
.UNINDENT
.INDENT 0.0
.IP \(bu 2
genindex
.IP \(bu 2
modindex
.IP \(bu 2
search
.UNINDENT
.SH AUTHOR
Brian Trammell
.SH COPYRIGHT
2013-2020, Brian Trammell
.\" Generated by docutils manpage writer.
.