.\" Text automatically generated by txt2man
.TH stenotype 1 "11 September 2020" "stenographer 1.0.1" ""
.SH NAME
\fBstenotype \fP- dump raw packets to disk
\fB
.SH SYNOPSIS
.nf
.fam C
\fBstenotype\fP [\fB-qv\fP?] [\fB--aiops\fP=NUM] [\fB--blocks\fP=NUM] [\fB--count\fP=NUM]
          [\fB--dir\fP=STRING] [\fB--fanout_id\fP=NUM] [\fB--fanout_type\fP=NUM]
          [\fB--fileage_sec\fP=NUM] [\fB--filesize_mb\fP=NUM] [\fB--filter\fP=STRING]
          [\fB--gid\fP=NUM] [\fB--iface\fP=STRING] [\fB--index_nicelevel\fP=NUM] [\fB--no_index\fP]
          [\fB--no_watchdogs\fP] [\fB--preallocate_file_mb\fP=NUM] [\fB--seccomp\fP=STRING]
          [\fB--threads\fP=NUM] [\fB--uid\fP=NUM] [\fB--help\fP] [\fB--usage\fP]

.fam T
.fi
.fam T
.fi
.SH DESCRIPTION
Stenotype is a mechanism for quickly dumping raw packets to disk. It aims to have a simple interface (no file rotation: that's left as an exercise for the reader) while being very powerful.
.PP
\fBstenotype\fP uses a NIC->disk pipeline specifically designed to provide as fast an output to disk as possible while just using the kernel's built-in mechanisms.
.IP 1. 4
NIC -> RAM: \fBstenotype\fP uses MMAP'd AF_PACKET with 1MB blocks and a high timeout to offload writing packets and deciding their layout to the kernel. The kernel packs all the packets it can into 1MB, then lets the userspace process know there's a block available in the MMAP'd ring buffer. Nicely, it guarantees no overruns (packets crossing the 1MB boundary) and good alignment to memory pages.
.IP 2. 4
RAM -> Disk: Since the kernel already gave us a single 1MB block of packets that's nicely aligned, we can O_DIRECT write it straight to disk. This avoids any additional copying or kernel buffering. To keep sequential reads going strong, we do all disk IO asynchronously via io_submit (which works specifically for O_DIRECT files\.\.\. joy!). Since the data is being written to disk asynchronously, we use the time it's writing to disk to do our own in-memory processing and indexing.
.PP
There are N (flag-specified) async IO operations available\.\.\. once we've used up all N, we block on a used one finishing, then reuse it. The whole pipeline consists of:
.RS
.IP \(bu 3
kernel gives userspace a 1MB block of packets
.IP \(bu 3
userspace iterates over packets in block, updates any indexes
.IP \(bu 3
userspace starts async IO operation to write block to disk
.IP \(bu 3
after N async IO operations are submitted, we synchronously wait for the
least recent one to finish.
.IP \(bu 3
when an async IO operation finishes, we release the 1MB block back to the
kernel to write more packets.
.SH OPTIONS
.TP
.B
\fB--aiops\fP=NUM
Max number of async IO operations
.TP
.B
\fB--blocks\fP=NUM
Total number of blocks to use, each is 1MB
.TP
.B
\fB--count\fP=NUM
Total number of packets to read, \fB-1\fP to read
forever
.TP
.B
\fB--dir\fP=STRING
Directory to store packet files in
.TP
.B
\fB--fanout_id\fP=NUM
If fanning out across processes, set this
.TP
.B
\fB--fanout_type\fP=NUM
TPACKET_V3 fanout type to fanout packets
.TP
.B
\fB--fileage_sec\fP=NUM
Files older than this many secs are rotated
.TP
.B
\fB--filesize_mb\fP=NUM
Max file size in MB before file is rotated
.TP
.B
\fB--filter\fP=STRING
BPF compiled filter used to filter which packets
will be captured. This has to be a compiled BPF in
hexadecimal, which can be obtained from a human
readable filter expression using the provided
compile_bpf.sh script.
.TP
.B
\fB--gid\fP=NUM
Drop privileges to this group
.TP
.B
\fB--iface\fP=STRING
Interface to read packets from
.TP
.B
\fB--index_nicelevel\fP=NUM
Nice level of indexing threads
.TP
.B
\fB--no_index\fP
Do not compute or write indexes
.TP
.B
\fB--no_watchdogs\fP
Don't start any watchdogs
.TP
.B
\fB--preallocate_file_mb\fP=NUM
When creating new files, preallocate to this
many MB
.TP
.B
\fB-q\fP
Quiet logging.  Each \fB-q\fP counteracts one \fB-v\fP
.TP
.B
\fB--seccomp\fP=STRING
Seccomp style, one of 'none', 'trace', 'kill'.
.TP
.B
\fB--threads\fP=NUM
Number of parallel threads to read packets with
.TP
.B
\fB--uid\fP=NUM
Drop privileges to this user
.TP
.B
\fB-v\fP
Verbose logging, may be given multiple times
.TP
.B
-?, \fB--help\fP
Give this help list
.TP
.B
\fB--usage\fP
Give a short usage message