\" Copyright (c) 2023, Peter Haag .\" All rights reserved. .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions are met: .\" .\" * Redistributions of source code must retain the above copyright notice, .\" this list of conditions and the following disclaimer. .\" * Redistributions in binary form must reproduce the above copyright notice, .\" this list of conditions and the following disclaimer in the documentation .\" and/or other materials provided with the distribution. .\" * Neither the name of the author nor the names of its contributors may be .\" used to endorse or promote products derived from this software without .\" specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" .\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE .\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE .\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR .\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF .\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS .\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN .\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE .\" POSSIBILITY OF SUCH DAMAGE. .\" .Dd $Mdocdate$ .Dt NFCAPD 1 .Os .Sh NAME .Nm nfcapd .Nd flow collector for netflow version v1, v5/v7 v9 and ipfix .Sh SYNOPSIS .Nm .Fl w Ar flowdir .Op Fl C Ar config .Op Fl z= .Op Fl D .Op Fl u Ar userid .Op Fl g Ar groupid .Op Fl S Ar num .Op Fl t Ar interval .Op Fl P Ar pidfile .Op Fl p Ar port .Op Fl d Ar device .Op Fl I Ar ident .Op Fl b Ar bindhost .Op Fl f Ar flowfile .Op Fl 4 .Op Fl 6 .Op Fl J Ar mcastgroup .Op Fl R Ar repeater .Op Fl A .Op Fl B Ar buffsize .Op Fl n Ar sourceparam .Op Fl M Ar multiflowdir .Op Fl s Ar rate .Op Fl i Ar metricrate .Op Fl m Ar metricpath .Op Fl e .Op Fl x Ar command .Op Fl X Ar extensionList .Op Fl E .Op Fl v .Op Fl V .Sh DESCRIPTION .Nm reads netflow data from the network and stores the records into binary formatted files. It accepts netflow v1, v5/v7, v9 and ipfix transparently. It is mostly compatible with a lot of other flow implementations such as cflow, jflow, pflow and accepts a wide range of exporters including CISCO Flexible Netflow (FNF), ASA firewalls and NAT devices for event logging. It has also support for a wide range of different vendors and their implementation of netflow, such as Juniper, VMware, PaloAlto devices and yaf. Sflow is a different technology. .Nm supports a large number of netflow v9 and ipfix elements according to the IANA assignments. .Pp If you want to collect sflow data, please have a look at .Ar sfcapd which is also part of the nfdump tools. .Pp .Nm also accepts pre-processed records from its companion collector nfpcapd. .Nm safes the flows in an output file, which is automatically rotated at a given interval - typically every 5min. These rotated output files are stored in the .Ar flowdir directory and are organized by timestamps. The output files are named according to the time interval in the following format: nfcapd.YYYYMMddhhmm e.g. nfcapd.202207110845 which contains flow data from July 11th 2022 08:45 onwards. If the rotation interval is set to a time, smaller then 60s, the naming extends to seconds e.g. nfcapd.20220711084510. .Pp .Nm can run in auto-expire mode .Fl e , which automatically expires old flow files, at the end of every rotation interval. .Ar nfexpire(1) explains in more details how to setup flow expiration. .Pp .Nm can run any given command .Fl x or shell script at the end of each rotation interval. .Pp .Nm can send universal flow metric information about the collected flow data (flow summary) to a UNIX socket. Programs, such as .Ar nfinflux or .Ar nfexporter may be used to send the metric information to an InfluxDB or to a Prometheus monitoring system. .Pp The options are as follows: .Bl -tag -width Ds .It Fl w Ar flowdir Set the flow directory to store the output files. If a sub hierarchy is specified with .Fl S the final directory is concatenated to flowdir/subdir. .It Fl C Ar config Reads additional configuration parameters from .Ar config file. .Nm tries to read the config file from the install default path .Ar $prefix/etc/ which may be overwritten by the environment variable .Ar NFCONF , which again is overwritten by this option .Fl C. If .Fl C Ar none is specified, then no config file is read, even if found in the search path. .It Fl p Ar portnum Set the port number to listen. Default port is 9995 .It Fl d Ar interface Reads flow data from an erspan encoded datalink. All traffic sent to this .Ar interface is interpreted as flow data stream. .It Fl b Ar bindhost Specifies the hostname/IPv4/IPv6 address to bind for listening. This can be an IP address or a hostname, resolving to a local IP address. .It Fl 4 Forces .Nm to listen on IPv4 addresses only. Can be used together with -b if a hostname has IPv4 and IPv6 addresses. .It Fl 6 Forces .Nm to listen on IPv6 addresses only. Can be used together with -b if a hostname has IPv4 and IPv6 addresses. .It Fl J Ar mcastgroup Join the specified IPv4 or IPv6 multicast group for listening. .It Fl R Ar host[/port] Enables the packet repeater. All incoming packets are sent additionally to another .Ar host and .Ar port . .Ar host is either a valid IPv4/IPv6 address, or a symbolic hostname, which resolves to a valid IP address. .Ar port may be omitted and defaults to 9995. Note: As IPv4/IPv6 are accepted the host/port separator is '/'. Up to 8 additional repeaters my be defined. Use this method to daisy chain collectors. .It Fl A Sets source address spoofing mode for the repeater. The source address of the repeated packages is set to the original IP address. This needs .Nm to be started with root privileges. Please note, that source spoofing may be blocked by firewalls or routers in your network. .It Fl I Ar ident Sets .Ar ident as identification string for the current source. This string is written into the output file to identify the source. Default is 'none'. If you have multiple sources, see option .Fl n below. .It Fl n Ar ident,IP,flowdir Configures a netflow source identified by the string .Ar ident, IP flowdir If you have multiple sources per collector, add multiple .Fl n options. All exporters send the flows to the same port .Fl p . Do not mix single source configuration .Fl I with multiple .Fl n options. .It Fl M Ar flowdir Set the flow directory for dynamic allocated exporters. New exporters are dynamically added when sending data. All exporters send netflow data to the same port and IP. For each dynamically added source, a new sub directory is created under .Ar flowdir with the name of the IP address of the exporter. All '.' and ':" in IP addresses are replaced be '-'. .Fl D Set daemon mode: fork to background and detach from terminal. .Nm terminates on signal TERM, INT or HUP. .It Fl P Ar pidfile Writes the running process ID into .Ar pidfilw . Use this option to integrate .Nm in start/stop files. .It Fl u Ar userid Drop privileges of running process to user .Ar userid . .Nm needs to be started as user root. .It Fl g Ar groupid Drop privileges of running process to group .Ar groupid . .Nm needs to be started as user root. .It Fl B Ar bufflen Sets the network socket input buffer to .Ar bufflen bytes. For high volume traffic it is recommended to raise this value to typically > 100k, otherwise you risk to lose packets. The default is OS (and kernel) dependent. .It Fl S Ar num Adds an additional directory sub hierarchy to store the data files. The default is 0, no sub hierarchy, which means all files go directly into .Ar flowdir . The .Ar flowdir is concatenated with the specified sub hierarchy format to create the final data directory. The following hierarchies are defined: .Bl -item -compact .It 0 default no hierarchy levels .It 1 %Y/%m/%d year/month/day .It 2 %Y/%m/%d/%H year/month/day/hour .It 3 %Y/%W/%u year/week_of_year/day_of_week .It 4 %Y/%W/%u/%H year/week_of_year/day_of_week/hour .It 5 %Y/%j year/day-of-year .It 6 %Y/%j/%H year/day-of-year/hour .It 7 %Y-%m-%d year-month-day .It 8 %Y-%m-%d/%H year-month-day/hour .El .It Fl t Ar interval Sets the time interval in seconds to rotate files. The default value is 300s ( 5min ). The smallest available interval is 2s. .It Fl s Ar rate Apply sampling rate .Ar rate to all netflow records, unless the sampling rate is announced by the exporting device. In that case the announced sampling rate is applied. If .Ar rate is negative, this will hard overwrite any device specific announced sampling rates. The sampling rate is used to multiply the number of packets and bytes in a record. Please note, this may vary from other volume counters such as SNMP etc. .It Fl z Compress flow files with LZO1X-1 compression. Fastest compression. .It Fl z=lzo Compress flow files with LZO1X-1 compression. Fastest compression. .It Fl z=lz4 Compress flow files with LZ4 compression. Fast and efficient. .It Fl z=bz2 Compress flow files with bz2 compression. Slow but most efficient. It is not recommended to use bz2 in a real time capturing. .It Fl e Sets auto-expire mode. At the end of every rotate interval .Fl t .Nm runs an expire cycle to delete files according to max lifetime and max filesize as defined by nfexpire(1) .It Fl x Ar command At the end of every .Fl t interval and after the file rotate has completed, .Nm runs the command .Ar command . The string for .Ar command may contain the following place holders, which are expanded before running: .Bl -item -compact .It %f File name of new data file including any sub hierarchy. .It %d Top .Ar flowdir . The full path of the new file is: %d/%f .It %t Time slot string in ISO format e.g. 201107110845. .It %u Time slot string in UNIX time format. .It %i Identification string .Ar ident string supplied by .Fl I .El .It Fl X Ar extensionList .Ar extensionList is a ',' separated list of extensions to be stored by .Nm . The numbers correspond to the extension list in nfxV3.h. By default extensions are added dynamically to store all data sent by the exporter. If .Ar extensionList is given, only those elements matching the extension are processed and stored. Usually this option is not needed, unless for specific requirements. .It Fl m Ar metricpath Enables the flow metric exporter. Flow metric information is sent to the UNIX socket .Ar metricpath at the rate specified by .Fl i This option may by used to export flow metric information to other systems such as InfluxDB or Prometheus. Please note: The flow metric does not include the full record. Only the flow statistics is sent. .It Fl i Ar metricrate Sets the interval for the flow metric exporter. This interval may be different from the file rotation interval .Ar t and is therefore independent from file rotation. .It Fl v Increase verbose level by 1. The verbose level may be increased for debugging purpose up to 3. .It Fl E Equal to -v -v -v. Print netflow records in block format to stdout. Please note, that not all elements are printed, which are available in the flow record. To inspect all elements, use .Ar nfdump .Fl o Ar raw This option is for debugging purpose only, to verify if incoming netflow data is processed correctly. .It Fl V Print .Nm version and exit. .It Fl h Print help text on stdout with all options and exit. .El .Sh RETURN VALUES .Nm returns 0 on success and 255 if initialization failed. .Sh SEE ALSO https://www.iana.org/assignments/ipfix/ipfix.xhtml .Pp https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html .Pp .Xr nfdump 1 .Xr nfpcapd 1 .Xr sfcapd 1 .Sh BUGS No software without bugs! Please report any bugs back to me.