table of contents
- NAME
- SYNOPSIS
- DESCRIPTION
- OPTIONS
- CONFIGURATION FILES
- PATTERNS, PATTERN TYPES AND MATCH VARIABLES
- CONTEXTS AND CONTEXT EXPRESSIONS
- ACTIONS, ACTION LISTS AND ACTION LIST VARIABLES
- PARSING ISSUES
- RULE TYPES
- EVENT CORRELATION OPERATIONS
- INPUT PROCESSING AND TIMING
- INTERNAL EVENTS AND CONTEXTS
- INTERPROCESS COMMUNICATION
- PERL INTEGRATION
- EXAMPLES
- ENVIRONMENT
- SIGNALS
- BUGS
- AUTHOR
- ACKNOWLEDGMENTS
- SEE ALSO
sec(1) | General Commands Manual | sec(1) |
NAME¶
sec - simple event correlatorSYNOPSIS¶
- sec
- [--conf=<file pattern> ...]
DESCRIPTION¶
SEC is an event correlation tool for advanced event processing which can be harnessed for event log monitoring, for network and security management, for fraud detection, and for any other task which involves event correlation. Event correlation is a procedure where a stream of events is processed, in order to detect (and act on) certain event groups that occur within predefined time windows. Unlike many other event correlation products which are heavyweight solutions, SEC is a lightweight and platform-independent event correlator which runs as a single process. The user can start it as a daemon, employ it in shell pipelines, execute it interactively in a terminal, run many SEC processes simultaneously for different tasks, and use it in a wide variety of other ways. SEC reads lines from files, named pipes, or standard input, matches the lines with patterns (regular expressions, Perl subroutines, etc.) for recognizing input events, and correlates events according to the rules in its configuration file(s). Rules are matched against input in the order they are given in the configuration file. If there are two or more configuration files, rule sequence from every file is matched against input (unless explicitly specified otherwise). SEC can produce output by executing external programs (e.g., snmptrap(1) or mail(1)), by writing to files, by sending data to TCP and UDP based servers, by calling precompiled Perl subroutines, etc. SEC can be run in various ways. For example, the following command line starts it as a daemon, in order to monitor events appended to the /var/log/messages syslog file with rules from /etc/sec/syslog.rules: /usr/bin/sec --detach --conf=/etc/sec/syslog.rules \--input=/var/log/messages Each time /var/log/messages is rotated, a new instance of /var/log/messages is opened and processed from the beginning. The following command line runs SEC in a shell pipeline, configuring it to process lines from standard input, and to exit when the /usr/bin/nc tool closes its standard output and exits: /usr/bin/nc -l 8080 | /usr/bin/sec --notail --input=- \
--conf=/etc/sec/my.conf Some SEC rules start event correlation operations, while other rules react immediately to input events or system clock. For example, suppose that SEC has been started with the following command line /usr/bin/sec --conf=/etc/sec/sshd.rules --input=/var/log/secure in order to monitor the /var/log/secure syslog file for sshd events. Also, suppose that the /etc/sec/sshd.rules configuration file contains the following rule for correlating SSH failed login syslog events: type=SingleWithThreshold
OPTIONS¶
- --conf=<file_pattern>
- expand <file_pattern> to filenames (with the Perl glob() function) and read event correlation rules from every file. Multiple --conf options can be specified at command line. Each time SEC receives a signal that forces a configuration reload, <file_pattern> is re-evaluated. See also INPUT PROCESSING AND TIMING section for a discussion on rule processing order for multiple configuration files.
- --input=<file_pattern>[=<context>]
- expand <file_pattern> to filenames (with the Perl glob() function) and use the files as input sources. An input file can be a regular file, named pipe, or standard input if - was specified. Multiple --input options can be specified at command line. Each time SEC receives the SIGHUP or SIGABRT signal, <file_pattern> is re-evaluated. If SEC experiences a system error when reading from an input file, it will close the file (use the --reopen-timeout option for reopening the file). If <context> is given, SEC will set up the context <context> each time it reads a line from input files that correspond to <file_pattern>. This will help the user to write rules that match data from particular input source(s) only. When there is an --input option with <context> specified, it will automatically enable the --intcontexts option. See INTERNAL EVENTS AND CONTEXTS section for more information.
- --input-timeout=<input_timeout>, --timeout-script=<timeout_script>
- if SEC has not observed new data in an input file during <input_timeout> seconds (or the file was closed <input_timeout> seconds ago), <timeout_script> will be executed with command line parameters 1 and <the name of the input file>. If fresh data become available again, <timeout_script> will be executed with command line parameters 0 and <the name of the input file>. Note that --input_timeout and --timeout_script options can be used as synonyms for --input-timeout and --timeout-script, respectively.
- --reopen-timeout=<reopen_timeout>
- if an input file is in the closed state (e.g., SEC fails to open the file at startup, because it has not been created yet), SEC will attempt to reopen the file after every <reopen_timeout> seconds until open succeeds. This option has no meaning when the --notail option is also specified. Note that --reopen_timeout is a synonym for --reopen-timeout.
- --check-timeout=<check_timeout>
- if SEC has not observed new data in an input file, the file will not be polled (both for status and data) during the next <check_timeout> seconds. Note that --check_timeout is a synonym for --check-timeout.
- --poll-timeout=<poll_timeout>
- a real number that specifies how many seconds SEC will sleep when no new data were read from input files. Default is 0.1 seconds. Note that --poll_timeout is a synonym for --poll-timeout.
- --socket-timeout=<socket_timeout>
- if a network connection to a remote peer can't be established within <socket_timeout> seconds, give up. Default is 60 seconds. Note that --socket_timeout is a synonym for --socket-timeout.
- --blocksize=<io_block_size>
- the number of bytes SEC will attempt to read at once from an input file. Default is 1024 (i.e., read from input files by 1KB blocks).
- --bufsize=<input_buffer_size>
- set all input buffers to hold last <input_buffer_size> lines. The content of input buffers will be compared with patterns that are part of rule definitions (i.e., no more than <input_buffer_size> lines can be matched by a pattern at a time). Default buffer size is 10 lines.
- --evstoresize=<event_store_size>
- set an upper limit to the number of events in context event stores.
- --cleantime=<clean_time>
- time interval in seconds that specifies how often internal event correlation and context lists are processed, in order to accomplish time-related tasks and to remove obsolete elements. See INPUT PROCESSING AND TIMING section for more information. Default is 1 second.
- --log=<logfile>
- use <logfile> for logging SEC activities. Note that if the SEC standard error is connected to a terminal, messages will also be logged there, in order to facilitate debugging.
- --syslog=<facility>
- use syslog for logging SEC activities. All messages will be logged with the facility <facility>, e.g., local0 (see syslog(3) for possible facility values). Warning: be careful with using this option if SEC is employed for monitoring syslog logfiles, because message loops might occur.
- --debug=<debuglevel>
- set logging verbosity for SEC. Setting debuglevel to <debuglevel>
means that all messages of level <debuglevel> and lower are logged
(e.g., if <debuglevel> is 3, messages from levels 1-3 are logged).
The following levels are recognized by SEC:
- --pid=<pidfile>
- SEC will store its process ID to <pidfile> at startup.
- --dump=<dumpfile>
- SEC will use <dumpfile> as its dumpfile. See SIGNALS section for more information. Default is /tmp/sec.dump.
- --quoting, --noquoting
- if the --quoting option is specified, operation description strings that are supplied to command lines of shellcmd, spawn, and cspawn actions will be put inside apostrophes. All apostrophes ' that strings originally contain will be masked. This option prevents the shell from interpreting special symbols that operation description strings might contain. Default is --noquoting.
- --tail, --notail
- if the --notail option is specified, SEC will process all data that are currently available in input files and exit after reaching all EOFs. If all input is received from a pipe and the --notail option is given, SEC terminates when the last writer closes the pipe (EOF condition). Please note that with named pipes --notail should be used with --norwfifo. With the --tail option, SEC will jump to the end of input files and wait for new lines to arrive. Each input file is tracked both by its name and i-node, and input file rotations are handled seamlessly. If the input file is recreated or truncated, SEC will reopen it and process its content from the beginning. If the input file is removed (i.e., there is just an i-node left without a name), SEC will keep the i-node open and wait for the input file recreation. Default is --tail.
- --fromstart, --nofromstart
- these flags have no meaning when the --notail option is also specified. When used in combination with --tail (or alone, since --tail is enabled by default), --fromstart will force SEC to read and process input files from the beginning to the end, before the 'tail' mode is entered. Default is --nofromstart.
- --detach, --nodetach
- if the --detach option is specified, SEC will disassociate itself from the controlling terminal and become a daemon at startup (note that SEC will close its standard input, standard output, and standard error, and change its working directory to the root directory). Default is --nodetach.
- --jointbuf, --nojointbuf
- if the --jointbuf option is specified, SEC uses joint input buffer for all input sources (the size of the buffer is set with the --bufsize option). The --nojointbuf option creates a separate input buffer for each input file, and a separate buffer for all synthetic and internal events (the sizes of all buffers are set with the --bufsize option). The --jointbuf option allows multiline patterns to match lines from several input sources, while the --nojointbuf pattern restricts the matching to lines from one input source only. See INPUT PROCESSING AND TIMING section for more information. Default is --jointbuf.
- --keepopen, --nokeepopen
- if the --keepopen option is specified, SEC will keep input files open across soft restarts. When the SIGABRT signal is received, SEC will not reopen input files which have been opened previously, but will only open input files which are in the closed state. The --nokeepopen option forces SEC to close and (re)open all input files during soft restarts. Default is --nokeepopen.
- --rwfifo, --norwfifo
- if the --norwfifo option is specified, named pipe input files are opened in read-only mode. In this mode, the named pipe has to be reopened when the last writer closes the pipe, in order to clear the EOF condition on the pipe. With the --rwfifo option, named pipe input files are opened in read-write mode, although SEC never writes to the pipes. In this mode, the pipe does not need to be reopened when an external writer closes it, since there is always at least one writer on the pipe and EOF will never appear. Therefore, if the --notail option has been given, --norwfifo should also be specified. Default is --rwfifo.
- --childterm, --nochildterm
- if the --childterm option is specified, SEC will send the SIGTERM signal to all its child processes when it terminates or goes through a full restart. Default is --childterm.
- --intevents, --nointevents
- SEC will generate internal events when it starts up, when it receives certain signals, and when it terminates gracefully. Specific rules can be written to match those internal events, in order to accomplish special tasks at SEC startup, restart, and shutdown. See INTERNAL EVENTS AND CONTEXTS section for more information. Default is --nointevents.
- --intcontexts, --nointcontexts
- SEC will create an internal context when it reads a line from an input file. This will help the user to write rules that match data from particular input source only. See INTERNAL EVENTS AND CONTEXTS section for more information. Default is --nointcontexts.
- --testonly, --notestonly
- if the --testonly option is specified, SEC will exit immediately after parsing the configuration file(s). If the configuration file(s) contained no faulty rules, SEC will exit with 0, otherwise with 1. Default is --notestonly.
- --help, -?
- SEC will output usage information and exit.
- --version
- SEC will output version information and exit.
CONFIGURATION FILES¶
Each SEC configuration file consists of rule definitions which are separated by empty lines, whitespace lines and/or comment lines. Each rule definition consists of keyword=value fields, one keyword and value per line. Values are case insensitive only where character case is not important (like the values specifying rule types, e.g., 'Single' and 'single' are treated identically). The backslash character (\) may be used at the end of a line to continue the current rule field in the next line. Lines which begin with the number sign (#) are treated as comments and ignored (whitespace characters may precede #). Any comment line, empty line, whitespace line, or end of file will terminate the preceding rule definition. For inserting comments into rule definitions, the rem keyword can be used. For example, the following lines define two rules: type=Singlethree consecutive A characters and writes the string \
"three A characters were observed" to standard output
- TakeNext
- after an event has matched the rule, search for matching rules in the configuration file will continue from the next rule.
- GoTo <label>
- after an event has matched the rule, search for matching rules will continue from the location of <label> in the configuration file (<label> must be defined with the label keyword anywhere in the configuration file *after* the current rule definition).
- DontCont (default value)
- after an event has matched the rule, search for matching rules ends in the *current* configuration file.
- EndMatch
- after an event has matched the rule, search for matching rules ends for *all* configuration files.
since 'continue' defaults to 'DontCont'
--conf=/etc/sec/sec1.rules --conf=/etc/sec/sec2.rules and the configuration file /etc/sec/sec1.rules has the following content: type=Single
PATTERNS, PATTERN TYPES AND MATCH VARIABLES¶
Patterns and pattern types are defined with pattern* and ptype* rule fields. Many pattern types define the number of lines N which the pattern matches (if N is omitted, 1 is assumed). If N is greater than 1, the scope of matching is set with the --jointbuf and --nojointbuf options. With --jointbuf, the pattern is used for matching N last input lines taken from the joint input buffer (the lines can come from different input sources). With --nojointbuf, the source of the last input line is identified, and the pattern is matched with N last input lines from the input buffer of the identified source.- SubStr[N]
- pattern is a string that is searched in the last N input lines L1, L2,
..., LN. If N is greater than 1, the input lines are joined into a string
"L1<NEWLINE>L2<NEWLINE>...<NEWLINE>LN", and
the pattern string will be searched from it. If the pattern string is
found in input line(s), the pattern matches. Backslash sequences \t, \n,
\r, \s, and \0 can be used in the pattern for denoting tabulation,
newline, carriage return, space character, and empty string, respectively,
while \\ denotes backslash itself. For example, consider the following
pattern definition:
ptype=substr
- RegExp[N]
- pattern is a Perl regular expression (see perlre(1) for more
information) for matching the last N input lines L1, L2, ..., LN. If N is
greater than 1, the input lines are joined into a string
"L1<NEWLINE>L2<NEWLINE>...<NEWLINE>LN", and
the regular expression is matched with this string. If the regular
expression matches, match variables will be set, and these match variables
can be used in other parts of the rule definition.
In addition to numbered match variables ($1, $2, etc.), SEC supports named
match variables $+{name} and the $0 variable. The $0 variable holds the
entire string of last N input lines that the regular expression has
matched. Named match variables can be created in newer versions of Perl
regular expression language, e.g., (?<myvar>AB|CD) sets $+{myvar} to
AB or CD. The special named match variable $+{_inputsrc} is created by SEC
and holds input file name(s) where matching line(s) came from.
For example, the following pattern matches the SSH "Connection
from" event, and sets $0 to the entire event line, both $1 and $+{ip}
to the IP address of the remote node, $2 to the port number at the remote
node, and $+{_inputsrc} to the input file name:
ptype=RegExp
- PerlFunc[N]
- pattern is a Perl function for matching the last N input lines L1, L2,
..., LN. The Perl function is compiled at SEC startup with the Perl
eval() function, and eval() must return a code reference for
the pattern to be valid (see also PERL INTEGRATION section). In order to
check whether the pattern matches input line(s), SEC will call the
function in list context and pass lines L1, L2, ..., LN and the names of
corresponding input files F1, F2, ..., FN to the function as parameters:
function(L1, L2, ..., LN, F1, F2, ..., FN)
Note that with the --nojointbuf option, the function is called with a
single file name parameter F, since lines L1, ..., LN are coming from the
same input file:
function(L1, L2, ..., LN, F)
Also note that if the input line is a synthetic event, the input file name
is Perl undefined value.
If the function returns several values or a single value that is true in
Perl boolean context, the pattern matches. If the function returns no
values or a single value that is false in Perl boolean context (0, empty
string or undefined value), the pattern does not match. If the pattern
matches, return values will be assigned to numbered match variables ($1,
$2, etc.). Like with RegExp patterns, the $0 variable is set to matching
input line(s), the $+{_inputsrc} variable is set to input file name(s),
and named match variables can be created from variable maps. For example,
consider the following pattern definition:
ptype=perlfunc2
- Cached
- pattern is a name that is searched in the pattern match cache (entries are
stored into the cache with the varmap* fields). If an entry with
the given name is found in the cache, the pattern matches, and match
variables and values are retrieved from the cache. For example, if the
input line matches the following pattern
ptype=perlfunc
- NSubStr[N]
- like SubStr[N], except that the result of the match is negated.
- NRegExp[N]
- like RegExp[N], except that the result of the match is negated and variable maps are not supported. Note that the only match variables set by the pattern are $0 and $+{_inputsrc}.
- NPerlFunc[N]
- like PerlFunc[N], except that the result of the match is negated and variable maps are not supported. Note that the only match variables set by the pattern are $0 and $+{_inputsrc}.
- NCached
- like Cached, except that the result of the match is negated.
- TValue
- pattern is a truth value, with TRUE and FALSE being legitimate values. TRUE always matches an input line, while FALSE never matches anything.
CONTEXTS AND CONTEXT EXPRESSIONS¶
A SEC context is a memory based entity which has one or more names, a lifetime, and an event store. Also, an action list can be set up for a context which is executed immediately before the context expires. For example, the action create MYCONTEXT 3600 (report MYCONTEXT /bin/mail root@localhost) creates the context MYCONTEXT which has a lifetime of 3600 seconds and empty event store. Also, immediately before MYCONTEXT expires and is dropped from memory, the action report MYCONTEXT /bin/mail root@localhost is executed which mails the event store of MYCONTEXT to root@localhost. Contexts can be used for event aggregation and reporting. Suppose the following actions are executed in this order: create MYCONTEXTACTIONS, ACTION LISTS AND ACTION LIST VARIABLES¶
Action lists are defined with the action* rule fields. An action list consists of action definitions that are separated by semicolons. Each action definition begins with a keyword specifying the action type. Depending on the action type, parameters may follow, and non-constant parameters may contain match variables. In order to use semicolons inside a non-constant parameter, the parameter must be enclosed in parentheses (the outermost set of parentheses will be removed by SEC during configuration file parsing). For example, the following action list consists of delete and shellcmd actions: delete MYCONTEXT; shellcmd (rm /tmp/sec1.tmp; rm /tmp/sec2.tmp) The delete action deletes the context MYCONTEXT, while the shellcmd action executes the command line rm /tmp/sec1.tmp; rm /tmp/sec2.tmp. Since the command line contains a semicolon, it has been enclosed in parentheses, since otherwise the semicolon would be mistakenly considered a separator between two actions. Apart from match variables, SEC supports action list variables in action lists which facilitate data sharing between actions and Perl integration. Each action list variable has a name which must begin with a letter and consist of letters, digits and underscores. In order to refer to an action list variable, its name must be prefixed by a percent sign (%). Unlike match variables, action list variables can only be used in action lists and they are substituted with their values immediately before the action list execution. Also, action list variables continue to exist after the current action list has been executed and can be employed in action lists of other rules. The following action list variables are predefined by SEC: %s - operation description string (the value of the desc field after match variables have been substituted with their values). Note that for the action2 field of Pair and PairWithWindow rules, the %s variable is set by evaluating the desc2 field of the rule. %t - current time in human-readable format (as returned by the date(1) utility). %u - current time in seconds since Epoch (as returned by the time(2) system call). Note that in order to disambiguate the variable from the following text, the variable name must be enclosed in braces (e.g., %{s}). Also, when action list variables are substituted with their values, each sequence "%%" is interpreted as a literal percent sign (%) which allows for masking the variables. For example, the string "s%%t" becomes "s%t" after substitution, not "s%<timestamp>". However, note that if %-prefixed match variables are supported for the action2 field of the Pair or PairWithWindow rule, the sequence "%%%" must be used in action2 for masking a variable, since the string goes through *two* variable substitution rounds (first for %-prefixed match variables and then for action list variables, e.g., the string "s%%%t" first becomes "s%%t" and finally "s%t"). Whenever a rule field goes through several substitution rounds, the $ or % characters are masked inside values substituted during earlier rounds, in order to avoid unwanted side effects during later rounds. If the action list variable has not been set, it is substituted with an empty string (i.e., a zero-width string). Thus the string "Value of A is: %a" becomes "Value of A is: " after substitution if the variable %a is unset. (Note that prior to SEC-2.6, unset variables were *not* substituted.) Finally, the values are substituted as strings, therefore values of other types (e.g., references) lose their original meaning, unless explicitly noted otherwise (e.g., if a Perl function reference is stored to an action list variable, the function can later be invoked through this variable with the call action). The following actions are supported (optional parameters are enclosed in square brackets):- none
- No action.
- logonly [<string>]
- Message <string> is logged to destinations given with the --log and --syslog options. The level of the log message is set to 4 (see the --debug option for more information on log message levels). Default value for <string> is %s.
- write <filename> [<string>]
- String <string> with a terminating newline is written to the file <filename> (<filename> may not contain whitespace). File may be a regular file, named pipe, or standard output (denoted by -). If the file is a regular file, <string> is appended to the end of the file. If the file does not exist, it is created as a regular file before writing. Note that the file will not be closed after the action completes, and the following write actions will access an already open file. However, several signals cause the file to be closed and reopened (see SIGNALS section for more information). Default value for <string> is %s.
- owritecl <filename> [<string>]
- Similar to the write action, except that the file <filename> is opened and closed at each write. Also, the string <string> is written without a terminating newline. If the file has already been opened by a previous write action, owritecl does not use existing filehandle, but opens and closes the file separately.
- udgram <filename> [<string>]
- String <string> is written to the UNIX datagram socket <filename> (<filename> may not contain whitespace). Note that the socket will not be closed after the action completes, and the following udgram actions will access an already open socket. However, several signals cause the socket to be closed and reopened (see SIGNALS section for more information). Default value for <string> is %s.
- ustream <filename> [<string>]
- String <string> is written to the UNIX stream socket <filename> (<filename> may not contain whitespace). Note that the socket will not be closed after the action completes, and the following ustream actions will access an already open socket. However, several signals cause the socket to be closed and reopened (see SIGNALS section for more information). Default value for <string> is %s.
- udpsock <host>:<port> [<string>]
- String <string> is sent to the UDP port <port> of the host <host>. Note that the UDP socket which is used for communication will not be closed after the action completes, and the following udpsock actions for the same remote peer will use an already existing socket. However, several signals cause the socket to be closed and recreated (see SIGNALS section for more information). Default value for <string> is %s.
- tcpsock <host>:<port> [<string>]
- String <string> is sent to the TCP port <port> of the host <host>. The timeout value given with the --socket-timeout option determines for how many seconds SEC will attempt to establish a connection to the remote peer. If the connection establishment does not succeed immediately, the tcpsock action buffers <string> in memory for later sending to the remote peer. Note that the relevant TCP socket will not be closed after <string> has been transmitted, and the following tcpsock actions for the same peer will use an already existing socket. However, several signals cause the socket to be closed and recreated (see SIGNALS section for more information). Default value for <string> is %s.
- shellcmd <cmdline>
- Fork a process for executing command line <cmdline>. If the --quoting option was specified and <cmdline> contains %s variables, the value of %s is quoted with apostrophes before substituting it into <cmdline>; note if the value of %s contains apostrophes, they are masked with backslashes (e.g., abc is converted to 'abc' and aa'bb is converted to 'aa'\''bb'). For additional information, see INTERPROCESS COMMUNICATION section.
- spawn <cmdline>
- Similar to the shellcmd action, except that each line from the standard output of <cmdline> becomes a synthetic event and will be treated like a line from input file (see the event action for more information). If the --intcontexts command line option is given, internal context _INTERNAL_EVENT is set up before each synthetic event is processed (see INTERNAL EVENTS AND CONTEXTS section for more information).
- cspawn <name> <cmdline>
- Similar to the spawn action, except that if the --intcontexts command line option is given, internal context <name> is set up for each synthetic event.
- pipe '[<string>]' [<cmdline>]
- Fork a process for executing command line <cmdline>. The string <string> with a terminating newline is written to the standard input of <cmdline> (apostrophes are used for disambiguating <string> from <cmdline>). If <string> contains semicolons, <string> must be enclosed in parentheses (e.g., pipe '($1;$2)' /bin/cat). Default value for <string> is %s. If <cmdline> is omitted, <string> is written to standard output. For additional information, see INTERPROCESS COMMUNICATION section.
- create [<name> [<time> [<action list>] ] ]
- Create a context with the name <name>, lifetime of <time> seconds, and empty event store. The <name> parameter may not contain whitespace and defaults to %s. The <time> parameter must evaluate to an unsigned integer at runtime. Specifying 0 for <time> or omitting the value means infinite lifetime. If <action list> is given, it will be executed when the context expires. If <action list> contains several actions, the list must be enclosed in parentheses. In <action list>, the internal context name _THIS may be used for referring to the current context (see INTERNAL EVENTS AND CONTEXTS section for a detailed discussion). If an already existing context is recreated with create, its remaining lifetime is set to <time> seconds, its action list is reinitialized, and its event store is emptied.
- delete [<name>]
- Delete the context <name>. The <name> parameter may not contain whitespace and defaults to %s.
- obsolete [<name>]
- Similar to the delete action, except that the action list of the context <name> (if present) is executed before deletion.
- set <name> <time> [<action list>]
- Change settings for the context <name>. The creation time of the context is set to the current time, and the lifetime of the context is set to <time> seconds. If the <action list> parameter is given, the context action list is set to <action list>. The <name> parameter may not contain whitespace and defaults to %s. The <time> parameter must evaluate to an unsigned integer or hyphen (-) at runtime. Specifying 0 for <time> means infinite lifetime. If <time> equals to -, the creation time and lifetime of the context are not changed. If <action list> contains several actions, the list must be enclosed in parentheses. In <action list>, the internal context name _THIS may be used for referring to the current context (see INTERNAL EVENTS AND CONTEXTS section for a detailed discussion).
- alias <name> [<alias>]
- Create an alias name <alias> for the context <name>. After creation, both <alias> and <name> will point to the same context data structure, and can thus be used interchangeably for referring to the context. The <name> and <alias> parameters may not contain whitespace, and <alias> defaults to %s. If the context <name> does not exist, the alias name is not created. If the delete action is called for one of the context names, the context data structure is destroyed, and all context names (which are now pointers to unallocated memory) cease to exist. Also note that when the context expires, its action list is executed only once, no matter how many names the context has.
- unalias [<alias>]
- Drop an existing context name <alias>, so that it can no longer be used for referring to the given context. The <alias> parameter may not contain whitespace and defaults to %s. If the name <alias> is the last reference to the context, the unalias action is identical to delete.
- add <name> [<string>]
- String <string> is appended to the end of the event store of the context <name>. The <name> parameter may not contain whitespace, and the <string> parameter defaults to %s. If the context <name> does not exist, the context is created with an infinite lifetime, empty action list and empty event store (as with create <name>) before adding the string to event store. If <string> is a multi-line string (i.e., it contains newlines), it is split into lines, and each line is appended to the event store separately.
- prepend <name> [<string>]
- Similar to the add action, except that the string <string> is prepended to the beginning of the event store of context <name>.
- fill <name> [<string>]
- Similar to the add action, except that the event store of the context <name> is emptied before <string> is added.
- report <name> [<cmdline>]
- Fork a process for executing command line <cmdline>, and write strings from the event store of the context <name> to the standard input of <cmdline>. Strings are written in the order they appear in the event store, with a terminating newline appended to each string. If the context <name> does not exist or its event store is empty, <cmdline> is not executed. The <name> parameter may not contain whitespace, and if <cmdline> is omitted, strings are written to standard output. For additional information, see INTERPROCESS COMMUNICATION section.
- copy <name> %<var>
- Strings s1,...,sn from the event store of the context <name> are joined into a multi-line string "s1<NEWLINE>...<NEWLINE>sn", and this string is assigned to the action list variable %<var>. If the context <name> does not exist, the value of %<var> does not change.
- empty <name> [%<var>]
- Similar to the copy action, except that the event store of the context <name> will be emptied after the assignment. If %<var> is omitted, the content of the event store is dropped without an assignment.
- pop <name> %<var>
- Remove the last string from the event store of context <name>, and assign it to the action list variable %<var>. If the event store is empty, %<var> is set to empty string. If the context <name> does not exist, the value of %<var> does not change.
- shift <name> %<var>
- Remove the first string from the event store of context <name>, and assign it to the action list variable %<var>. If the event store is empty, %<var> is set to empty string. If the context <name> does not exist, the value of %<var> does not change.
- exists %<var> <name>
- If the context <name> exists, set the action list variable %<var> to 1, otherwise set %<var> to 0.
- getsize %<var> <name>
- Find the number of strings in the event store of context <name>, and assign this number to the action list variable %<var>. If the context <name> does not exist, %<var> is set to Perl undefined value.
- getaliases %<var> <name>
- Find all alias names for context <name>, join the names into a multi-line string "alias1<NEWLINE>...<NEWLINE>aliasn", and assign this string to the action list variable %<var>. If the context <name> does not exist, the value of %<var> does not change.
- getltime %<var> <name>
- Find the lifetime of context <name>, and assign this number to the action list variable %<var>. If the context <name> does not exist, the value of %<var> does not change.
- getctime %<var> <name>
- Find the creation time of context <name>, and assign this number to the action list variable %<var>. If the context <name> does not exist, the value of %<var> does not change.
- setctime <time> <name>
- Set the creation time of context <name> to <time>. The <time> parameter must evaluate to seconds since Epoch (as reported by the time(2) system call), and must reflect a time moment between the previous creation time and the current time (both endpoints included).
- event [<time>] [<string>]
- After <time> seconds, create a synthetic event <string>. If <string> is a multi-line string (i.e., it contains newlines), it is split into lines, and from each line a separate synthetic event is created. SEC will treat each synthetic event like a line from an input file -- the event will be matched against rules and it might trigger further actions. If the --intcontexts command line option is given, internal context _INTERNAL_EVENT is set up for synthetic event(s) (see INTERNAL EVENTS AND CONTEXTS section for more information). The <time> parameter is an integer constant. Specifying 0 for <time> or omitting the value means "now". Default value for <string> is %s.
- tevent <time> [<string>]
- Similar to the event action, except that the <time> parameter may contain variables and must evaluate to an unsigned integer at runtime.
- cevent <name> <time> [<string>]
- Similar to the tevent action, except that if the --intcontexts command line option is given, internal context <name> is set up for synthetic event(s).
- reset [<offset>] [<string>]
- Terminate event correlation operation(s) with the operation description string <string>. Note that the reset action works only for operations started from the same configuration file. The <offset> parameter is used to refer to a specific rule in the configuration file. If <offset> is given, the operation started by the given rule is terminated (if it exists). If <offset> is an unsigned integer N, it refers to the N-th rule in the configuration file. If <offset> is 0, it refers to the current rule. If <offset> begins with the plus (+) or minus (-) sign, it specifies an offset from the current rule (e.g., -1 denotes the previous and +1 the next rule). If <offset> is not given, SEC checks for each rule from the current configuration file if an operation with <string> has been started by this rule, and the operation is terminated if it exists. Default value for <string> is %s. For additional information, see EVENT CORRELATION OPERATIONS section.
- getwpos %<var> <offset> [<string>]
- Find the beginning of the event correlation window for an event correlation operation, and set the action list variable %<var> to this timestamp. The value assigned to %<var> is measured in seconds since Epoch (as reported by the time(2) system call). As with the reset action, the event correlation operation is identified by the operation description string <string> and the rule offset <offset>. If the operation does not exist, the value of %<var> does not change. Default value for <string> is %s. For additional information, see EVENT CORRELATION OPERATIONS section.
- setwpos <time> <offset> [<string>]
- Set the beginning of the event correlation window to <time> for an event correlation operation (if it exists). The <time> parameter must evaluate to seconds since Epoch (as reported by the time(2) system call), and must reflect a time moment between the previous window position and the current time (both endpoints included). As with the reset action, the event correlation operation is identified by the operation description string <string> and the rule offset <offset>. Default value for <string> is %s. For additional information, see EVENT CORRELATION OPERATIONS section.
- assign %<var> [<string>]
- Assign string <string> to the action list variable %<var>. Default value for <string> is %s.
- free %<var>
- Unset the action list variable %<var>.
- eval %<var> <code>
- The parameter <code> is a Perl miniprogram that is compiled and executed by calling the Perl eval() function in the Perl list context. If the miniprogram returns a single value, it is assigned to the action list variable %<var>. If the miniprogram returns several values s1,...,sn, they are joined into a multi-line string "s1<NEWLINE>...<NEWLINE>sn", and this string is assigned to %<var>. If no value is returned, %<var> is set to Perl undefined value. If eval() fails, the value of %<var> does not change. Since most Perl programs contain semicolons which are also employed by SEC as action separators, it is recommended to enclose the <code> parameter in parentheses, in order to mask the semicolons in <code>. For additional information, see PERL INTEGRATION section.
- call %<var> %<ref> [<paramlist>]
- Call the precompiled Perl function referenced by the action list variable %<ref>, and assign the result to the action list variable %<var>. The %<ref> parameter must be a code reference that has been previously created with the eval action. The <paramlist> parameter (if given) is a string which specifies parameters for the function. The parameters must be separated by whitespace in the <paramlist> string. If the function returns a single value, it is assigned to %<var>. If the function returns several values s1,...,sn, they are joined into a multi-line string "s1<NEWLINE>...<NEWLINE>sn", and this string is assigned to %<var>. If no value is returned, %<var> is set to Perl undefined value. If the function encounters a fatal runtime error or %<ref> is not a code reference, the value of %<var> does not change. For additional information, see PERL INTEGRATION section.
- lcall %<var> [<paramlist>] -> <code>
- Call the precompiled Perl function <code> and assign the result to the action list variable %<var>. The <code> parameter must be a valid Perl function definition that is compiled at SEC startup with the Perl eval() function, and eval() must return a code reference. The <paramlist> parameter (if given) is a string which specifies parameters for the function. The parameters must be separated by whitespace in the <paramlist> string. If the function returns a single value, it is assigned to %<var>. If the function returns several values s1,...,sn, they are joined into a multi-line string "s1<NEWLINE>...<NEWLINE>sn", and this string is assigned to %<var>. If no value is returned, %<var> is set to Perl undefined value. If the function encounters a fatal runtime error, the value of %<var> does not change. Since most Perl functions contain semicolons which are also employed by SEC as action separators, it is recommended to enclose the <code> parameter in parentheses, in order to mask the semicolons in <code>. For additional information, see PERL INTEGRATION section.
- rewrite <lnum> [<string>]
- Replace last <lnum> lines in the input buffer with string <string>. If the --nojointbuf option was specified and the action is triggered by a matching event, the action modifies the buffer which holds this event. If the --nojointbuf option was specified and the action is triggered by the system clock (e.g., the action is executed from the Calendar rule), the action modifies the buffer which holds the last already processed event. With the --jointbuf option, the content of the joint input buffer is rewritten. The <lnum> parameter must evaluate to an unsigned integer at runtime. If <lnum> evaluates to 0, <lnum> is reset to the number of lines in <string>. If the value of <lnum> is greater than the buffer size N, <lnum> is reset to N. If <string> contains less than <lnum> lines, <string> will be padded with leading empty lines. If <string> contains more than <lnum> lines, only leading <lnum> lines from <string> are written into the buffer. Default value for <string> is %s. For additional information, see INPUT PROCESSING AND TIMING section.
- if %<var> ( <action list> ) [ else ( <action list2> ) ]
- If the action list variable %<var> evaluates true in the Perl boolean context (i.e., it holds a defined value which is neither 0 nor empty string), execute the action list <action list>. If the second action list <action list2> is given with the optional else-statement, it is executed if %<var> either does not exist or evaluates false (i.e., %<var> holds 0, empty string or Perl undefined value).
- while %<var> ( <action list> )
- Execute the action list <action list> repeatedly until the action list variable %<var> evaluates true in the Perl boolean context (i.e., it holds a defined value which is neither 0 nor empty string).
- break
- If used inside a while-loop, terminates its execution; otherwise terminates the execution of the entire action list.
- continue
- If used inside a while-loop, starts the next iteration of the loop; otherwise terminates the execution of the entire action list.
PARSING ISSUES¶
As already noted, SEC context expressions and action lists may contain parentheses which are used for grouping and masking purposes. When SEC parses its configuration, it checks whether parentheses in context expressions and action lists are balanced (i.e., whether each parenthesis has a counterpart), since unbalanced parentheses introduce ambiguity. This can cause SEC to reject some legitimate constructs, e.g., action=eval %o (print ")";) is considered an invalid action list (however, note thatRULE TYPES¶
This section provides a detailed discussion of SEC rule types.SINGLE RULE¶
The Single rule immediately executes an action list when an event has matched the rule. An event matches the rule if the pattern matches the event and the context expression (if given) evaluates TRUE. The Single rule supports the following fields:- type
- fixed to Single (value is case insensitive, so single or sIngLe can be used instead).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive).
- ptype
- pattern type (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map.
- context (optional)
- context expression.
- desc
- operation description string.
- action
- action list.
- rem (optional, may appear more than once)
- remarks and comments.
(report ftp_$1 /bin/mail root@localhost) type=single
delete ftp_$1 This ruleset is created for monitoring the ftpd log file. The first rule creates the context ftp_<pid> when someone connects from host ristov2 over FTP and establishes a new ftp session (the session is identified by the PID of the process which has been created for handling this session). The second rule adds all further log file lines for the session <pid> to the event store of the context ftp_<pid> (before adding a line, the rule checks if the context exists). After adding a line, the rule extends context's lifetime for 30 minutes and sets the action list that will be executed when the context expires. The third rule mails collected log file lines to root@localhost when the session <pid> is closed. Collected lines will also be mailed when the session <pid> has been inactive for 30 minutes (no log file lines observed for that session). Note that the log file line that has matched the first rule is also matched against the second rule (since the first rule has the continue field set to TakeNext). Since the second rule always matches this line, it will become the first line in the event store of ftp_<pid>. The second rule has also its continue field set to TakeNext, since otherwise no log file lines would reach the third rule.
SINGLEWITHSCRIPT RULE¶
The SingleWithScript rule forks a process for executing an external program when an event has matched the rule. The names of all currently existing contexts are written to the standard input of the program. After the program has been forked, the rule matching continues immediately, and the program status will be checked periodically until the program exits. If the program returns 0 exit status, the action list defined by the action field is executed; otherwise the action list defined by the action2 field is executed (if given). The SingleWithScript rule supports the following fields:- type
- fixed to SingleWithScript (value is case insensitive).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive).
- ptype
- pattern type (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map.
- context (optional)
- context expression.
- script
- an external program.
- desc
- operation description string.
- action
- action list.
- action2 (optional)
- action list.
- rem (optional, may appear more than once)
- remarks and comments.
SINGLEWITHSUPPRESS RULE¶
The SingleWithSuppress rule runs event correlation operations for filtering repeated instances of the same event during T seconds. The value of T is defined by the window field. When an event has matched the rule, SEC evaluates the operation description string given with the desc field. If the operation for the given string and rule does not exist, SEC will create it with the lifetime of T seconds, and the operation immediately executes an action list. If the operation exists, it consumes the matching event without any action. The SingleWithSuppress rule supports the following fields:- type
- fixed to SingleWithSuppress (value is case insensitive).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive).
- ptype
- pattern type (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map.
- context (optional)
- context expression.
- desc
- operation description string.
- action
- action list.
- window
- event correlation window size (value is an integer constant).
- rem (optional, may appear more than once)
- remarks and comments.
PAIR RULE¶
The Pair rule runs event correlation operations for processing event pairs during T seconds. The value of T is defined by the window field. Default value is 0 which means infinity. When an event has matched the conditions defined by the pattern and context field, SEC evaluates the operation description string given with the desc field. If the operation for the given string and rule exists, it consumes the matching event without any action. If the operation does not exist, SEC will create it with the lifetime of T seconds, and the operation immediately executes an action list defined by the action field. SEC will also copy the match conditions given with the pattern2 and context2 field into the operation, and substitute match variables with their values in copied conditions. If the event does not match conditions defined by the pattern and context field, SEC will check the match conditions of all operations started by the given rule. Each matching operation executes the action list given with the action2 field and finishes. The Pair rule supports the following fields:- type
- fixed to Pair (value is case insensitive).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive). Specifies the point-of-continue after a match by pattern and context.
- ptype
- pattern type for pattern (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map for pattern.
- context (optional)
- context expression, evaluated together with pattern.
- desc
- operation description string.
- action
- action list.
- continue2 (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive). Specifies the point-of-continue after a match by pattern2 and context2.
- ptype2
- pattern type for pattern2 (value is case insensitive).
- pattern2
- pattern.
- varmap2 (optional)
- variable map for pattern2.
- context2 (optional)
- context expression, evaluated together with pattern2.
- desc2
- format string that sets the %s variable for action2.
- action2
- action list.
- window (optional)
- event correlation window size (value is an integer constant).
- rem (optional, may appear more than once)
- remarks and comments.
PAIRWITHWINDOW RULE¶
The PairWithWindow rule runs event correlation operations for processing event pairs during T seconds. The value of T is defined by the window field. When an event has matched the conditions defined by the pattern and context field, SEC evaluates the operation description string given with the desc field. If the operation for the given string and rule exists, it consumes the matching event without any action. If the operation does not exist, SEC will create it with the lifetime of T seconds. SEC will also copy the match conditions given with the pattern2 and context2 field into the operation, and substitute match variables with their values in copied conditions. If the event does not match conditions defined by the pattern and context field, SEC will check the match conditions of all operations started by the given rule. Each matching operation executes the action list given with the action2 field and finishes. If the operation has not observed a matching event by the end of its lifetime, it executes the action list given with the action field before finishing. The PairWithWindow rule supports the following fields:- type
- fixed to PairWithWindow (value is case insensitive).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive). Specifies the point-of-continue after a match by pattern and context.
- ptype
- pattern type for pattern (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map for pattern.
- context (optional)
- context expression, evaluated together with pattern.
- desc
- operation description string.
- action
- action list.
- continue2 (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive). Specifies the point-of-continue after a match by pattern2 and context2.
- ptype2
- pattern type for pattern2 (value is case insensitive).
- pattern2
- pattern.
- varmap2 (optional)
- variable map for pattern2.
- context2 (optional)
- context expression, evaluated together with pattern2.
- desc2
- format string that sets the %s variable for action2.
- action2
- action list.
- window
- event correlation window size (value is an integer constant).
- rem (optional, may appear more than once)
- remarks and comments.
SINGLEWITHTHRESHOLD RULE¶
The SingleWithThreshold rule runs event correlation operations for counting repeated instances of the same event during T seconds, and taking an action if N events are observed. The values of T and N are defined by the window and thresh field, respectively. When an event has matched the rule, SEC evaluates the operation description string given with the desc field. If the operation for the given string and rule does not exist, SEC will create it with the lifetime of T seconds. The operation will memorize the occurrence time of the event (current time as returned by the time(2) system call), and compare the number of memorized occurrence times with the threshold N. If the operation has observed N events, it executes the action list defined by the action field, and consumes all further matching events without any action. If the rule has an optional action list defined with the action2 field, the operation will execute it before finishing, provided that the action list given with action has been previously executed by the operation. Note that a sliding window is employed for event counting -- if the operation has observed less than N events by the end of its lifetime, it drops occurrence times which are older than T seconds, and extends its lifetime for T seconds from the earliest remaining occurrence time. If there are no remaining occurrence times, the operation finishes without executing an action list. The SingleWithThreshold rule supports the following fields:- type
- fixed to SingleWithThreshold (value is case insensitive).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive).
- ptype
- pattern type (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map.
- context (optional)
- context expression.
- desc
- operation description string.
- action
- action list.
- action2 (optional)
- action list.
- window
- event correlation window size (value is an integer constant).
- thresh
- counting threshold (value is an integer constant).
- rem (optional, may appear more than once)
- remarks and comments.
SINGLEWITH2THRESHOLDS RULE¶
The SingleWith2Thresholds rule runs event correlation operations which take action if N1 events have been observed in the window of T1 seconds, and then at most N2 events will be observed in the window of T2 seconds. The values of T1, N1, T2, and N2 are defined by the window, thresh, window2, and thresh2 field, respectively. When an event has matched the rule, SEC evaluates the operation description string given with the desc field. If the operation for the given string and rule does not exist, SEC will create it with the lifetime of T1 seconds. The operation will memorize the occurrence time of the event (current time as returned by the time(2) system call), and compare the number of memorized occurrence times with the threshold N1. If the operation has observed N1 events, it executes the action list defined by the action field, and starts another counting round for T2 seconds. If no more than N2 events have been observed by the end of the window, the operation executes the action list defined by the action2 field and finishes. Note that both windows are sliding -- the first window slides like the window of the SingleWithThreshold operation, while the beginning of the second window is moved to the second earliest memorized event occurrence time when the threshold N2 is violated. The SingleWith2Thresholds rule supports the following fields:- type
- fixed to SingleWith2Thresholds (value is case insensitive).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive).
- ptype
- pattern type (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map.
- context (optional)
- context expression.
- desc
- operation description string.
- action
- action list.
- window
- event correlation window size (value is an integer constant).
- thresh
- counting threshold.
- desc2
- format string that sets the %s variable for action2.
- action2
- action list.
- window2
- event correlation window size (value is an integer constant).
- thresh2
- counting threshold.
- rem (optional, may appear more than once)
- remarks and comments.
EVENTGROUP RULE¶
The EventGroup rule runs event correlation operations for counting repeated instances of N different events e1,...,eK during T seconds, and taking an action if threshold conditions c1,...,cN for *all* events are satisfied (i.e., for each event eK there are at least cK event instances in the window). The values for N and T are defined by the type and window field, respectively, while values for c1,...,cN are given with the thresh* fields. Values for N and c1,...,cN default to 1. Note that the event correlation window is sliding like the window of the SingleWithThreshold operation. When an event has matched the rule, SEC evaluates the operation description string given with the desc field. If the operation for the given string and rule does not exist, SEC will create it with the lifetime of T seconds. The operation will memorize the occurrence time of the event (current time as returned by the time(2) system call), and compare the number of memorized occurrence times for each eK with the threshold cK. If all threshold confitions are satisfied, the operation executes the action list defined by the action field, and consumes all further matching events without re-executing the action list if the multact field is set to No (this is the default). However, if multact is set to Yes, the operation will evaluate the threshold conditions on every further matching event, executing the action list given with the action field if all conditions are satisfied, and sliding the event correlation window forward when the window is about to expire (if no events remain in the window, the operation will finish). If the rule definition has an optional action list defined with the count* field for event eK, the operation executes it every time an instance of eK is observed (even if multact is set to No and the operation has already executed the action list given with action). If the rule definition has an optional action list defined with the init field, the operation executes it immediately after the operation has been created. If the rule definition has an optional action list defined with the end field, the operation executes it immediately before the operation finishes. Note that this action list is *not* executed when the operation is terminated with the reset action. If the rule definition has an optional action list defined with the slide field, the operation executes it immediately after the event correlation window has slidden forward. However, note that moving the window with the setwpos action will *not* trigger the execution. The EventGroup rule supports the following fields:- type
- EventGroup[N] (value is case insensitive, N defaults to 1).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive). Specifies the point-of-continue after a match by pattern and context.
- ptype
- pattern type for pattern (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map for pattern.
- context (optional)
- context expression, evaluated together with pattern.
- count (optional)
- action list for execution after a match by pattern and context.
- thresh (optional)
- counting threshold for events matched by pattern and context (value is an integer constant, default is 1).
- ...
- continueN (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive). Specifies the point-of-continue after a match by patternN and contextN.
- ptypeN
- pattern type for patternN (value is case insensitive).
- patternN
- pattern.
- varmapN (optional)
- variable map for patternN.
- contextN (optional)
- context expression, evaluated together with patternN.
- countN (optional)
- action list for execution after a match by patternN and contextN.
- threshN (optional)
- counting threshold for events matched by patternN and contextN (value is an integer constant, default is 1).
- desc
- operation description string.
- action
- action list.
- init (optional)
- action list.
- end (optional)
- action list.
- slide (optional)
- action list.
- multact (optional)
- Yes or No (values are case insensitive, default is No).
- window
- event correlation window size (value is an integer constant).
- rem (optional, may appear more than once)
- remarks and comments.
SUPPRESS RULE¶
The Suppress rule takes no action when an event has matched the rule, and keeps matching events from being processed by later rules in the configuration file. The Suppress rule supports the following fields:- type
- fixed to Suppress (value is case insensitive).
- ptype
- pattern type (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map.
- context (optional)
- context expression.
- desc (optional)
- string for describing the rule.
- rem (optional, may appear more than once)
- remarks and comments.
create SUPPRESS_IP_$2 3600
CALENDAR RULE¶
The Calendar rule was designed for executing actions at specific times. Unlike all other rules, this rule reacts only to the system clock, ignoring other input. The Calendar rule executes the action list given with the action field if the current time matches all conditions of the time specification given with the time field. The action list is executed only once for any matching minute. The rule employs a time specification which closely resembles the crontab(1) style, but there are some subtle differences. The time specification consists of five or six conditions separated by whitespace. The first condition matches minutes (allowed values are 0-59), the second condition matches hours (allowed values are 0-23), the third condition days (allowed values are 0-31, with 0 denoting the last day of the month), the fourth condition months (allowed values are 1-12), and the fifth condition weekdays (allowed values are 0-7, with 0 and 7 denoting Sunday). The sixth condition is optional and matches years (allowed values are 0-99 which denote the last two digits of the year). Asterisks (*), ranges of numbers (e.g., 8-11), and lists (e.g., 2,5,7-9) are allowed as conditions. Asterisks and ranges may be augmented with step values (e.g., 47-55/2 means 47,49,51,53,55). Note that unlike crontab(1) time specification, the day and weekday conditions are *not* joined with logical OR, but rather with logical AND. Therefore, 0 1 25-31 10 7 means 1AM on last Sunday in October. On the other hand, with crontab(1) the same specification means 1AM in every last seven days or every Sunday in October. Also, unlike some versions of cron(8), SEC is not restricted to take action only during the first second of the current minute (for example, if SEC is started at the 22th second of a minute, the wildcard condition produces a match for this minute). The Calendar rule supports the following fields:- type
- fixed to Calendar (value is case insensitive).
- time
- time specification.
- context (optional)
- context expression.
- desc
- operation description string.
- action
- action list.
- rem (optional, may appear more than once)
- remarks and comments.
JUMP RULE¶
The Jump rule submits matching events to specific ruleset(s) for further processing. If the event matches the rule, SEC continues the search for matching rules in configuration file set(s) given with the cfset field. Rules from every file are tried in the order of their appearance in the file. Configuration file sets can be created from Options rules with the joincfset field, with each set containing at least one configuration file. If more that one set name is given with cfset, sets are processed from left to right; a matching rule in one set doesn't prevent SEC from processing the following sets. If the constset field is set to Yes, set names are assumed to be constants and will not be searched for match variables at runtime. The Jump rule supports the following fields:- type
- fixed to Jump (value is case insensitive).
- continue (optional)
- TakeNext, DontCont, EndMatch or GoTo <label> (apart from <label>, values are case insensitive).
- ptype
- pattern type (value is case insensitive).
- pattern
- pattern.
- varmap (optional)
- variable map.
- context (optional)
- context expression.
- cfset (optional)
- configuration file set names that are separated by whitespace.
- constset (optional)
- Yes or No (values are case insensitive, default is Yes).
- desc (optional)
- string for describing the rule.
- rem (optional, may appear more than once)
- remarks and comments.
OPTIONS RULE¶
The Options rule sets processing options for the ruleset in the current configuration file. If more than one Options rule is present in the configuration file, the last instance overrides all previous ones. Note that the Options rule is only processed when SEC (re)starts and reads in the configuration file. Since this rule is not applied at runtime, it can never match events, react to the system clock, or start event correlation operations. The joincfset field lists the names of one or more configuration file sets, and the current configuration file will be added to each set. If a set doesn't exist, it will be created and the current configuration file becomes its first member. If the procallin field is set to No, the rules from the configuration file will be used for matching input from Jump rules only. The Options rule supports the following fields:- type
- fixed to Options (value is case insensitive).
- joincfset (optional)
- configuration file set names that are separated by whitespace.
- procallin (optional)
- Yes or No (values are case insensitive, default is Yes).
- rem (optional, may appear more than once)
- remarks and comments.
EVENT CORRELATION OPERATIONS¶
In order to identify event correlation operations, SEC assigns an ID to every operation that is composed from the configuration file name, the rule number, and the operation description string (defined by the desc field of the rule). If there are N rules in the configuration file, the rule numbers belong to the range 0..N-1, and the number of the k-th rule is k-1. Note that since the configuration file name and rule number are part of the operation ID, different rules can have identical desc fields without a danger of a clash between operations. For example, if the configuration file my.conf contains only one rule type=SingleWithThresholdINPUT PROCESSING AND TIMING¶
SEC processes input data iteratively by reading one line at each iteration, writing this line into a relevant input buffer, and matching the content of the updated buffer with rules from configuration files. If during the matching process an action list is executed which creates new input events (e.g., through the event action), they are *not* written to buffer(s) immediately, but rather consumed at following iterations. Note that when both synthetic events and regular input are available for processing, synthetic events are always consumed first. When all synthetic events have been consumed iteratively, SEC will start processing new data from input files. With the --jointbuf option, SEC employs a joint input buffer for all input sources which holds N last input lines (the value of N can be set with the --bufsize option). Updating the input buffer means that the new line becomes the first element of the buffer, while the last element (the oldest line) is removed from the end of the buffer. With the --nojointbuf option, SEC maintains a buffer of N lines for each input file, and if the input line comes from file F, the buffer of F is updated as described previously. There is also a separate buffer for synthetic and internal events. Suppose SEC is started with the following command line /usr/bin/sec --conf=/etc/sec/test-multiline.conf --jointbuf \--input=/var/log/prog1.log --input=/var/log/prog2.log and the configuration file /etc/sec/test-multiline.conf has the following content: type=Single
line contains "test1" and the second line "test2", and \
writes the matching lines to standard output
--input=- --nojointbuf and the configuration file /etc/sec/test-rewrite.conf has the following content: type=Single
line contains "test1" and the second line "test2", and \
joins these lines in the input buffer
and writes this line to standard output
INTERNAL EVENTS AND CONTEXTS¶
In the action list of a context, the context can also be referred with the internal context name _THIS. The name _THIS is created and deleted dynamically by SEC and it points to the context only during its action list execution. This feature is useful when the context has had several names during its lifetime (created with the alias action), and it is hard to determine which names exist when the context expires. For example, if the context is created with create A 60 (report A /bin/mail root) which is immediately followed by alias A B and unalias A, the report action will fail since the name A no longer refers to the context. However, replacing the first action with create A 60 (report _THIS /bin/mail root) will produce the correct result. If the --intevents command line option is given, SEC will generate internal events when it is started up, when it receives certain signals, and when it terminates normally. Inside SEC, internal event is treated as if it was a line that was read from a SEC input file. Specific rules can be written to match internal events, in order to take some action (e.g., start an external event correlation module with spawn when SEC starts up). The following internal events are supported: SEC_STARTUP - generated when SEC is started (this event will always be the first event that SEC sees) SEC_RESTART - generated after SEC has received the SIGHUP signal and all internal data structures have been cleared (this event will be the first event that SEC sees after reloading its configuration) SEC_SOFTRESTART - generated after SEC has received the SIGABRT signal (this event will be the first event that SEC sees after reloading its configuration) SEC_LOGROTATE - generated after SEC has received the SIGUSR2 signal (this event will be the first event that SEC sees after reopening its log file and closing its outputs) SEC_SHUTDOWN - generated when SEC receives the SIGTERM signal, or when SEC reaches all EOFs of input files after being started with the --notail option. With the --childterm option, SEC sleeps for 3 seconds after generating SEC_SHUTDOWN event, and then sends SIGTERM to its child processes (if a child process was triggered by SEC_SHUTDOWN, this delay leaves the process enough time for setting a signal handler for SIGTERM). Before generating an internal event, SEC sets up a context named SEC_INTERNAL_EVENT, in order to disambiguate internal events from regular input. The SEC_INTERNAL_EVENT context is deleted immediately after the internal event has been matched against all rules. If the --intcontexts command line option is given, or there is an --input option with a context specified, SEC creates an internal context each time it reads a line from an input file or a synthetic event. The internal context is deleted immediately after the line has been matched against all rules. For all input files that have the context name explicitly set with --input=<file_pattern>=<context>, the name of the internal context is <context>. If the line was read from the input file <filename> for which there is no context name set, the name of the internal context is _FILE_EVENT_<filename>. For synthetic events, the name of the internal context defaults to _INTERNAL_EVENT, but cspawn and cevent actions can be used for generating synthetic events with custom internal context names. This allows for writing rules that match data from one particular input source only. For example, the rule type=Suppress--input=/var/log/messages=MESSAGES \
--input=/var/log/secure=SECURE \
--input=/var/log/cron=CRON and the rule file /etc/sec/my.conf contains the following rules type=Single
INTERPROCESS COMMUNICATION¶
The SingleWithScript rule and shellcmd, spawn, cspawn, pipe, and report actions fork a child process for executing an external program. If the program command line contains shell metacharacters, the command line is first parsed by the shell which then starts the program. SEC communicates with its child processes through pipes (created with the pipe(2) system call). When the child process is at the read end of the pipe, data have to be written to the pipe in blocking mode which ensures reliable data transmission. In order to avoid being blocked, SEC forks another child process for writing data to the pipe reliably. After forking an external program, SEC continues immediately, and checks the program status periodically until the program exits. The running time of a child process is not limited in any way. With the --childterm option, SEC sends the SIGTERM signal to all child processes when it terminates. If some special exit procedures need to be accomplished in the child process (or the child wishes to ignore SIGTERM), then the child must install a handler for the SIGTERM signal. Note that if the program command line contains shell metacharacters, the parsing shell will run as a child process of SEC and the parent process of the program. Therefore, the SIGTERM signal will be sent to the shell, *not* the program. In order to avoid this, the shell's builtin exec command can be used (see sh(1) for more information) which replaces the shell with the program without forking a new process, e.g., action=spawn exec /usr/local/bin/myscript.pl 2>/var/log/myscript.log Note that if an action list includes two actions which fork external programs, the execution order these programs is not determined by the order of actions in the list, since both programs are running asynchronously. In order to address this issue, the execution order must be specified explicitly (e.g., instead of writing action=shellcmd cmd1; shellcmd cmd2, use the shell && operator and write action=shellcmd cmd1 && cmd2). Sometimes it is desireable to start an external program and provide it with data from several rules. In order to create such setup, names pipes can be used. For example, if /var/log/pipe is a named pipe, then action=shellcmd /usr/bin/logger -f /var/log/pipe -p user.notice starts the /usr/bin/logger utility which sends all lines read from /var/log/pipe to the local syslog daemon with the "user" facility and "notice" level. In order to feed events to /usr/bin/logger, the write action can be used (e.g., write /var/log/pipe This is my event). Although SEC keeps the named pipe open across different write actions, the pipe will be closed on the reception of SIGHUP, SIGABRT and SIGUSR2 signals. Since a number of command line tools terminate on receiving EOF from input, they need restarting after such signals have arrived. For this purpose, the --intevents option and SEC internal events can be used. For example, the following rule starts the /usr/bin/logger utility at SEC startup, and also restarts it after the reception of relevant signals: type=Singleshellcmd /usr/bin/logger -f /var/log/pipe -p user.notice Note that if /var/log/pipe is never opened for writing by a write action, /usr/bin/logger will never see EOF and will thus not terminate. The owritecl action opens and closes /var/log/pipe, in order to ensure the presence of EOF in such cases.
PERL INTEGRATION¶
SEC supports patterns, context expressions, and actions which involve calls to the Perl eval() function or the execution of precompiled Perl code. The use of Perl code in SEC patterns and context expressions allows for creating proper match conditions for scenarios which can't be handled by a simple regular expression match. For example, consider the following iptables syslog events: May 27 10:00:15 box1 kernel: iptables: IN=eth0 OUT= MAC=08:00:27:be:9e:2f:00:10:db:ff:20:03:08:00 SRC=10.6.4.14 DST=10.1.8.2 LEN=84 TOS=0x00 PREC=0x00 TTL=251 ID=61426 PROTO=ICMP TYPE=8 CODE=0 ID=11670 SEQ=2if ($line !~ /kernel: iptables:/g) { return 0; } \
while ($line =~ /\G\s*([A-Z]+)(?:=(\S*))?/g) { \
$var{$1} = defined($2)?$2:1; \
} return \%var; }
exists($_[0]->{"FIN"}) ; } )
if %ret ( logonly Storabe loaded ) else ( eval %o exit(1) ) type=Single
%{Storable::retrieve("/tmp/SEC_CONTEXTS")}; } ) type=Single
Storable::store(\%main::context_list, "/tmp/SEC_CONTEXTS"); } ) However, note that modifying data structures within SEC code is recommended only for advanced users who have carefully studied relevant parts of the code. Finally, sometimes larger chunks of Perl code have to be used for event processing and correlation. However, writing many lines of code directly into a rule is cumbersome and may decrease its readability. In such cases it is recommended to separate the code into a custom Perl module which is loaded at SEC startup, and use the code through the module interface (see perlmod(1) for further details): type=Single
if %ret ( none ) else ( eval %o exit(1) ) type=Single
EXAMPLES¶
Example 1 - a ruleset for Cisco events¶
This section presents an example rulebase for managing Cisco devices. It is assumed that the managed devices have syslog logging enabled, and that all syslog messages are sent to a central host and written to logfile(s) that are monitored by SEC. # Set up contexts NIGHT and WEEKEND for nightsExample 2 - hierarchically organized rulesets for iptables and sshd events¶
This section presents an example of hierarchically organized rules for processing Linux iptables events from /var/log/messages and SSH login events from /var/log/secure. It is assumed that all rule files reside in the /etc/sec directory and that the rule hierarchy has two levels. The file /etc/sec/main.rules contains first-level Jump rules for matching and parsing events from input files and submitting them to proper rulesets for further processing. All other rule files in the /etc/sec directory contain second-level rules which receive their input from first-level Jump rules. Also, the example assumes that SEC is started with the following command line: /usr/bin/sec --conf=/etc/sec/*.rules --intcontexts \--input=/var/log/messages --input=/var/log/secure #
if ($line !~ /kernel: iptables:/g) { return 0; } \
while ($line =~ /\G\s*([A-Z]+)(?:=(\S*))?/g) { \
$var{$1} = defined($2)?$2:1; \
} return \%var; }
exists($_[0]->{"FIN"}) ; } ) \
&& !SUPPRESS_IP_$+{SRC}
create SUPPRESS_IP_$+{SRC} 3600
!exists($_[0]->{"ACK"}) ; } ) \
&& !SUPPRESS_IP_$+{SRC}
create SUPPRESS_IP_$+{SRC} 3600
$_[0]->{"srcport"} < 1024 && \
defined($_[0]->{"invuser"}); } )
defined($_[0]->{"invuser"}); } )
( sub { return $_[0]->{"status"} eq "Accepted"; } ) && \
$+{user} %+{user} $+{srcip} %+{srcip} -> \
( sub { return $_[0] eq $_[1] && $_[2] eq $_[3]; } )
ENVIRONMENT¶
If the SECRC environment variable is set, SEC expects it to contain the name of its resource file. Resource file lines which are empty or which begin with the number sign (#) are ignored (whitespace may precede #). Each remaining line is appended to the argv array of SEC as a *single* element. Also, the lines are appended to argv in the order they appear in the resource file. Therefore, if the SEC command line option has a value, the option name and the value must either be separated by the equal sign (=) or a newline. Here is a simple resource file example: # read events from standard inputSIGNALS¶
- SIGHUP
- full restart -- SEC will reinterpret its command line and resource file options, reopen its log and input files, close its output files and sockets (these will be reopened on demand), reload its configuration, and drop *all* event correlation state (all event correlation operations will be terminated, all contexts will be deleted, all action list variables will be erased, etc.). With the --childterm option, SEC will also send the SIGTERM signal to its child processes.
- SIGABRT
- soft restart -- SEC will reinterpret its command line and resource file options, reopen its log file, and close its output files and sockets (these will be reopened on demand). If the --keepopen option is specified, previously opened input files will remain open across soft restart, otherwise all input files will be reopened. SEC will (re)load configuration from rule files which have been modified (file modification time returned by stat(2) has changed) or created after the previous configuration load. SEC will also terminate event correlation operations started from rule files that have been modified or removed after the previous configuration load. Other operations and previously loaded configuration from unmodified rule files will remain intact. Note that on some systems SIGIOT is used in place of SIGABRT.
- SIGUSR1
- detailed information about the current state of SEC (performance and rule matching statistics, running event correlation operations, created contexts, etc.) will be written to the SEC dumpfile.
- SIGUSR2
- SEC will reopen its logfile (useful for logfile rotation), and also close its output files and sockets which will be reopened on demand.
- SIGINT
- SEC will increase its logging level by one; if the current level is 6, the level will be set back to 1. Please note this feature is available only if SEC is running non-interactively (e.g., in daemon mode).
- SIGTERM
- SEC will terminate gracefully. With the --childterm option, all SEC child processes will receive SIGTERM.
BUGS¶
With some locale settings, apostrophes (') in this man page might be displayed incorrectly. As a workaround, set the LANG environment variable to C when reading this man page (e.g., env LANG=C man sec).AUTHOR¶
Risto Vaarandi (ristov at users d0t s0urcef0rge d0t net)ACKNOWLEDGMENTS¶
This work is supported by SEB. The author thanks the following people for supplying software patches, documentation fixes, and suggesting new features: Al Sorrell, David Lang, James Brown, Jon Frazier, Mark D. Nagel, Peter Eckel, Rick Casey, and William Gertz. Last but not least, the author expresses his profound gratitute to John P. Rouillard for many great ideas and creative discussions that have helped to develop SEC.SEE ALSO¶
cron(8), crontab(1), date(1), fork(2), mail(1), perl(1), perlmod(1), perlre(1), pipe(2), sh(1), snmptrap(1), stat(2), syslog(3), time(2)July 2014 | SEC 2.7.6 |