'\"macro stdmacro
.\"
.\" Copyright (c) 2014-2020,2022 Red Hat.
.\"
.\" This program is free software; you can redistribute it and/or modify it
.\" under the terms of the GNU General Public License as published by the
.\" Free Software Foundation; either version 2 of the License, or (at your
.\" option) any later version.
.\"
.\" This program is distributed in the hope that it will be useful, but
.\" WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
.\" or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
.\" for more details.
.\"
.\"
.TH PMFETCHGROUP 3 "PCP" "Performance Co-Pilot"
.SH NAME
\f3pmCreateFetchGroup\f1,
\f3pmExtendFetchGroup_item\f1,
\f3pmExtendFetchGroup_indom\f1,
\f3pmExtendFetchGroup_event\f1,
\f3pmExtendFetchGroup_timestamp\f1,
\f3pmExtendFetchGroup_timespec\f1,
\f3pmExtendFetchGroup_timeval\f1,
\f3pmFetchGroup\f1,
\f3pmGetFetchGroupContext\f1,
\f3pmClearFetchGroup\f1,
\f3pmDestroyFetchGroup\f1 \- simplified performance metrics value fetch and conversion
.SH "C SYNOPSIS"
.ft 3
#include <pcp/pmapi.h>
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmCreateFetchGroup(pmFG *\fIptr\fP, int \fItype\fP, const char *\fIname\fP);
.br
.ti -8n
int pmExtendFetchGroup_item(pmFG \fIpmfg\fP, const char *\fImetric\fP, const char *\fIinstance\fP, const char *\fIscale\fP, pmAtomValue *\fIout_value\fP, int \fIout_type\fP, int \fIout_sts\fP);
.br
.ti -8n
int pmExtendFetchGroup_indom(pmFG \fIpmfg\fP, const char *\fImetric\fP, const char *\fIscale\fP, int \fIout_inst_codes\fP[], char *\fIout_inst_names\fP[], pmAtomValue \fIout_values\fP[], int \fIout_type\fP, int \fIout_stss\fP[], unsigned int \fIout_maxnum\fP, unsigned int *\fIout_num\fP, int *\fIout_sts\fP);
.br
.ti -8n
int pmExtendFetchGroup_event(pmFG \fIpmfg\fP, const char *\fImetric\fP, const char *\fIinstance\fP, const char *\fIfield\fP, const char *\fIscale\fP, struct timespec \fIout_times\fP[], pmAtomValue \fIout_values\fP[], int \fIout_type\fP, int \fIout_stss\fP[], unsigned int \fIout_maxnum\fP, unsigned int *\fIout_num\fP, int *\fIout_sts\fP);
.br
.ti -8n
int pmExtendFetchGroup_timestamp(pmFG \fIpmfg\fP, struct timeval *\fIout_value\fP);
.br
.ti -8n
int pmExtendFetchGroup_timespec(pmFG \fIpmfg\fP, struct timespec *\fIout_value\fP);
.br
.ti -8n
int pmExtendFetchGroup_timeval(pmFG \fIpmfg\fP, struct timeval *\fIout_value\fP);
.br
.ti -8n
int pmGetFetchGroupContext(pmFG \fIpmfg\fP);
.br
.ti -8n
int pmFetchGroup(pmFG \fIpmfg\fP);
.br
.ti -8n
int pmClearFetchGroup(pmFG \fIpmfg\fP);
.br
.ti -8n
int pmDestroyFetchGroup(pmFG \fIpmfg\fP);
.sp
.in
.hy
.ad
cc ... \-lpcp
.ft 1
.SH DESCRIPTION
The fetchgroup functions implement a registration-based mechanism to
fetch groups of performance metrics, including automation for general
unit, rate and type conversions as well as convenient instance and value
encodings.
They constitute a powerful and compact alternative to the
classic Performance Metrics Application Programming Interface (\c
.BR PMAPI (3))
sequence of separate lookup, check, fetch, iterate, extract and
convert functions.
.PP
The general idea consists of two stages.
In the setup stage, the
application identifies metrics of interest by name and with desired
conversions, and register a unique \fBpmAtomValue\fP output location
where the fetchgroup system is to later deposit the result.
It is also possible to identify a metric with an instance domain, and
register a unique \fIvector\fP of pmAtomValue objects for them.
In the operation stage, one simple \fBpmFetchGroup\fP function fetches,
decodes, converts, and stores all metrics to their destinations, where
the application can \fIread\fP them directly.
This function may be called repeatedly, and each time
new \fBpmAtomValue\fP values will be stored in the same destinations.
Rate conversions between consecutive samples may be requested.
.PP
Each fetchgroup is associated with a private PMAPI context, so it can
manipulate instance profiles and other such state without disrupting
other contexts.  The instance profile is manipulated to optimize
fetches of individual items, even if some are derived metrics.
This private PMAPI context belongs to the fetchgroup,
is used for all of its internal operations, and will be destroyed.
.PP
Multiple fetchgroups may be used concurrently, independently.
An opaque type \fBpmFG\fP is used to identify a fetchgroup, which is
passed to all related function calls.
.SS Creating a fetchgroup
.ft 3
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmCreateFetchGroup(pmFG *\fIptr\fP, int \fItype\fP, const char *\fIname\fP);
.sp
.in
.hy
.ad
.ft 1
This function creates a new fetchgroup, associated with a new PMAPI
context.
The \fItype\fP and \fIname\fP parameters are relayed to
.BR pmNewContext (3)
for creation of the context.
The fetchgroup identifier is returned upon success through
the \fIptr\fP pointer.
This object is later used as a parameter to all other fetchgroup
functions.
The private PMAPI context may be accessed with
\fBpmGetFetchGroupContext\fP, if required.
.PP
The normal function return code is zero, and \fIptr\fP is set.
This function may fail in case of \fBpmNewContext\fP or memory
allocation errors.
Those are indicated with a negative return code and a cleared \fIptr\fP value.
.SS Getting the private PMAPI context
.ft 3
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmGetFetchGroupContext(pmFG \fIpmfg\fP);
.sp
.in
.hy
.ad
.ft 1
This function returns the private PMAPI context used by the given
fetchgroup.
It may be safely used to adjust some configuration
parameters of the context, such as via
.BR pmSetMode (3),
before fetchgroup extension and fetching begins.
.PP
However, \fImutation\fP of this context by PMAPI functions after
this time may disrupt fetchgroup functionality.
For example, a
\fBpmSetMode\fP call could invalidate one rate-conversion time-step.
.PP
The normal function return code is the context number.
.SS Extending a fetchgroup with a metric instance of interest
.ft 3
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmExtendFetchGroup_item(pmFG \fIpmfg\fP, const char *\fImetric\fP, const char *\fIinstance\fP, const char *\fIscale\fP, pmAtomValue *\fIout_value\fP, int \fIout_type\fP, int *\fIout_sts\fP);
.sp
.in
.hy
.ad
.ft 1
This function registers interest in a single metric and optional instance.
The metric name is given in the mandatory \fImetric\fP parameter, which is
checked immediately via
.BR pmLookupName (3)
and other calls.
If and only if the
metric has an instance domain, the specific instance of interest may
be named by the \fIinstance\fP parameter, which is checked immediately
via
.BR pmNameInDom (3);
otherwise pass NULL.
If the fetchgroup context
is a set of archives, it is possible that the metric / instance pair is not
yet defined at the current time origin.
Therefore, this function may
attempt to seek to the \fIend\fP of the current set of archives
\fItemporarily\fP to retry the metric / instance lookup.
.PP
The optional \fIscale\fP parameter specifies desired unit/scale/rate
conversions for the metric value.
It can take the following values:
.IP NULL 4
No unit/scale conversion.
If metric has \fBPM_SEM_COUNTER\fP semantics,
perform rate conversion.
.IP "rate" 4
Perform rate conversion regardless of semantics, and no unit/scale conversion.
.IP "instant" 4
Perform no rate conversion regardless of semantics, and no unit/scale conversion.
.IP "EXPRESSION" 4
Perform unit/scale/rate conversion as specified by the EXPRESSION,
which is parsed by
.BR pmParseUnitsStr (3).
This may be useful to
assert a canonical scaling for the resulting metric value, independent
of PCP version or configuration.
Dimensionality must match the
metric, except if rate conversion is requested, in which case the time
dimension must be one smaller than the metric's time dimension.
Note that
the type of rate conversion performed here matches the
.BR rate(x)
function in derived metric expressions, in that it is calculated as the
naive difference between previous and current values of a metric, divided
by elapsed time.
For example, if a counter wraps around, or a non-counter
value decreases, a \fInegative\fP output rate may be computed.
.PP
The optional but usual \fIout_value\fP parameter specifies the
\fBpmAtomValue\fP where the converted result should later be stored.
If the value is NULL, fetching and conversions will be attempted, and
possible errors reported, but the result tossed away.
The mandatory
\fIout_type\fP parameter specifes the \fBPM_TYPE_*\fP requested for
the output value.
It need not match the metric's native type, as the
fetchgroup facility is capable of casting between all supported types
(including to and from strings).
.PP
Any errors subsequently encountered during fetching, unit/scale/rate
conversion, or casting, will result in the assignment of a sentinel
value to the output \fBpmAtomValue\fP (see the ``UNUSUAL SITUATIONS''
section below).
In addition, if the optional \fIout_sts\fP parameter
is specified, an appropriate PMAPI error code will be stored there.
.PP
As a review, only the \fIpmfg\fP, \fImetric\fP, and \fIout_type\fP
parameters are mandatory.
Others may be NULL to indicate applicaton disinterest.
.PP
The normal function return code is zero.
This function may fail in
case of various lookup, type- and conversion- checking errors.
Those are indicated with a negative return code.
.SS Extending a fetchgroup with a metric instance domain of interest
.ft 3
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmExtendFetchGroup_indom(pmFG \fIpmfg\fP, const char* \fImetric\fP, const char *\fIscale\fP, int \fIout_inst_codes\fP[], char *\fIout_inst_names\fP[], pmAtomValue \fIout_values\fP[], int \fIout_type\fP, int \fIout_stss\fP[], unsigned int \fIout_maxnum\fP, unsigned int *\fIout_num\fP, int *\fIout_sts\fP);
.sp
.in
.hy
.ad
.ft 1
This function generalizes the \fBpmExtendFetchGroup_item\fP function
by registering interest in a whole instance domain.
Therefore, the
function registers preallocated \fIvectors\fP for output variables
(instead of a singleton).
Instances will be stored in sorted order in
elements of those vectors.
The concepts are otherwise the same.
.PP
The metric name is specified by the mandatory \fImetric\fP parameter.
Note that it \fImay\fP refer to a metric without an instance domain,
in which case the single output value will appear as one unnamed
instance.
.PP
The optional \fIscale\fP parameter specifies desired unit/scale/rate
conversions for the metric value, same as above.
.PP
The optional \fIout_inst_codes\fP parameter specifies a vector of
integers, where the raw instance number of the fetched metrics should
later be stored.
.PP
The optional \fIout_inst_names\fP parameter specifies a vector of
strings, where the instance names of the fetched metrics should later
be stored.
If an instance does not have a corresponding name, a NULL
pointer is stored instead.
The application must not modify or
.BR free (3)
strings in that vector.
.PP
The optional \fIout_values\fP parameter specifies a vector of
\fBpmAtomValue\fP objects where the converted result should later be
stored.
The mandatory \fIout_type\fP parameter specifies the
\fBPM_TYPE_*\fP requested for the all output values, same as above.
.PP
The optional \fIout_stss\fP parameter specifies a vector of integers
where per-instance error codes should be stored.
.PP
The mandatory \fIout_maxnum\fP parameter specifies the number of
elements of the vectors above.
In other words, it tells the
fetchgroup the maximum number of instances which are expected.
The optional \fIout_num\fP parameter specifies an integer where the
actual number of instances should later be stored.
It will range between 0 and \fIout_maxnum\fP.
It is initialized to 0 by this function.
.PP
Finally, the optional \fIout_sts\fP parameter specifies a single
location where an integer status code for the overall fetch for this
metric should be stored.
Normally, this will be zero.
Other than a severe fetch error, one may see a \fBPM_ERR_TOOBIG\fP here
if the number of instances actually encountered was larger than
\fIout_maxnum\fP.
.PP
Any errors subsequently encountered during fetching, unit/scale/rate
conversion, or casting, will result in the assignment of a sentinel
value to the appropriate output \fBpmAtomValue\fP (see
the ``UNUSUAL SITUATIONS'' section below).
In addition, if the optional
\fIout_stss\fP parameter was specified, a PMAPI error code will be
stored in the appropriate position.
.PP
As a review, only the \fIpmfg\fP, \fImetric\fP, \fIout_type\fP, and
\fIout_maxnum\fP parameters are mandatory.
Others may be NULL to indicate applicaton disinterest.
.PP
The normal function return code is zero.
This function may fail in
case of various lookup, type- and conversion- checking errors.
Those are indicated with a negative return code.
.SS Extending a fetchgroup with an event field
.ft 3
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmExtendFetchGroup_event(pmFG \fIpmfg\fP, const char *\fImetric\fP, const char *\fIinstance\fP, const char *\fIfield\fP, const char *\fIscale\fP, struct timespec \fIout_times\fP[], pmAtomValue \fIout_values\fP[], int \fIout_type\fP, int \fIout_stss\fP[], unsigned int \fIout_maxnum\fP, unsigned int *\fIout_num\fP, int *\fIout_sts\fP);
.sp
.in
.hy
.ad
.ft 1
This function registers interest in all instances of one field of all
records of an event metric.
Since event metrics may return multiple
records per fetch, and each record may have multiple fields of a given
field metric type, this function registers preallocated \fIvectors\fP
for output variables, similarly to \fBpmExtendFetchGroup_indom\fP.
They are filled in temporal/sequential order.
.PP
The metric name is specified by the mandatory \fImetric\fP parameter.
It must be of \fBPM_TYPE_EVENT\fP.
If the metric has an instance
domain, the \fIinstance\fP parameter is mandatory to identify the
instance of interest.
.PP
The field to extract from event records is specified by the mandatory
\fIfield\fP parameter, which is a metric name of normal scalar type.
As is typical for event field metrics, it should not have an instance
domain.
The optional \fIscale\fP parameter specifies desired
unit/scale conversions on this metric value.
Rate conversions are
\fBnot available\fP, because of ambiguity about which previous value
to compute rates from.
.PP
The optional \fIout_times\fP parameter specifies a vector of
\fBtimespec\fP structs, which will receive a copy of the timestamp
of the event record where each particular field was found.
.PP
The optional \fIout_values\fP parameter specifies a vector of
\fBpmAtomValue\fP objects where the converted result should later
be stored.
The mandatory \fIout_type\fP parameter specifies the
\fBPM_TYPE_*\fP requested for the output values.
.PP
The optional \fIout_stss\fP parameter specifies a vector of integers
where per-field error codes should be stored.
.PP
The mandatory \fIout_maxnum\fP parameter specifies the number of
elements of the vectors above.
In other words, it tells the
fetchgroup the maximum number of instances which are expected.
The
optional \fIout_num\fP parameter specifies an integer where the the
actual number of instances should later be stored.
It will range
between zero and \fIout_maxnum\fP.
It is initialized to zero by this function.
.PP
Finally, the optional \fIout_sts\fP parameter specifies a single
location where an integer status code for the overall fetch for this
metric should be stored.
Normally, this will be zero, even if no
event field values were found (\fIout_num\fP would then be zero).
Other than a severe fetch error, one may see a \fBPM_ERR_TOOBIG\fP
here if the number of fields actually encountered was larger than
\fIout_maxnum\fP.
.PP
Any errors subsequently encountered during fetching, unit/scale
conversion, or casting, will result in the assignment of a sentinel
value to the appropriate output \fBpmAtomValue\fP (see
the ``UNUSUAL SITUATIONS'' section below).
In addition, if the optional
\fIout_stss\fP parameter was specified, a PMAPI error code will be
stored in the appropriate position.
.PP
As a review, only the \fIpmfg\fP, \fImetric\fP, \fIfield\fP,
\fIout_type\fP, and \fIout_maxnum\fP parameters are mandatory.
Others may be NULL to indicate applicaton disinterest.
.PP
The normal function return code is zero.
This function may fail in
case of various lookup, type- and conversion- checking errors.
Those are indicated with a negative return code.
.SS Extending a fetchgroup with the fetch timestamp
.ft 3
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmExtendFetchGroup_timestamp(pmFG \fIpmfg\fP, struct timeval *\fIout_value\fP);
.br
.ti -8n
int pmExtendFetchGroup_timespec(pmFG \fIpmfg\fP, struct timespec *\fIout_value\fP);
.br
.ti -8n
int pmExtendFetchGroup_timeval(pmFG \fIpmfg\fP, struct timeval *\fIout_value\fP);
.sp
.in
.hy
.ad
.ft 1
These functions register interest in the \fBpmHighResResult\fP timestamp.
If the \fIout_value\fP pointer is non-NULL, at every future
\fBpmFetchGroup\fR call, the corresponding result timestamp will be
copied there.
.SS Fetching all metrics in a fetchgroup
.ft 3
.sp
.ad l
.hy 0
.in +8n
.ti -8n
int pmFetchGroup(pmFG \fIpmfg\fP);
.sp
.in
.hy
.ad
.ft 1
This function performs one \fBpmFetch\fP on its private PMAPI context,
including all the metrics that were registered via prior
\fBpmExtendFetchGroup_*\fP calls.
It runs all the data extraction and
conversion operations necessary to populate all the requested output
variables.
.PP
The normal function return code is zero or positive, as per the
underlying \fBpmFetch\fP function.
This function may fail in
case of severe fetch errors, which are indicated with a negative
return code.
.PP
In the case of per-metric availability or conversion errors, or severe
fetch errors, output variables are reset to sentinel values and
individual error codes are set.
\fIPM_ERR_AGAIN\fP signals
rate-conversion failure due to lack of a previous value.
.PP
However, temporarily absent metrics with discrete semantics are exempt
from some sentinel/error processing: if a \fBpmFetchGroup\fP fails to
collect a result for a discrete metric (pmHighResResult
pmValueSet.numval==0), then the last seen valid value (if any) is
retained.
This is intended to ease the processing of sets of archives with a
mixture of once- and repeatedly-sampled metrics.
.SS Clearing a fetchgroup
.ft 3
.nf
int pmClearFetchGroup(pmFG \fIpmfg\fP);
.fi
.ft 1
.PP
When the current fetch state of a fetchgroup is no longer needed,
it may be explicitly reset with this function.
It releases any dynamically stored state but keeps the private PMAPI
context intact for subsequent use (i.e. no change to the context is
made at all and the context remains at the current fetch offset).
It frees any pointers such as indom instance names or strings that
may have been stored in output variables.
.SS Destroying a fetchgroup
.ft 3
.nf
int pmDestroyFetchGroup(pmFG \fIpmfg\fP);
.fi
.ft 1
.PP
When the fetchgroup is no longer needed, it may be explicitly freed
with this function.
It releases any dynamically stored state, as well
as the private PMAPI context.
It clears frees any pointers such as
indom instance names or strings that may have been stored in output
variables.
.SH EXAMPLE
The following program demonstrates fetchgroup usage.
Run it with
different $PCP_DISK_UNITS environment variables to see different
unit/rate conversion in effect.
.PP
.\" NB: the following code escapes \ for nroff
.nf
#include <pcp/pmapi.h>
#include <stdio.h>

#define pcpassert(sts) \\
    while (sts<0) { fprintf(stderr, "%s\\n", pmErrStr(sts)); exit(42); }

int main()
{
    pmFG fg;
    pmAtomValue v, v2;
    enum { v3_maxnum = 100 };
    pmAtomValue v3_values[v3_maxnum];
    char *v3_names[v3_maxnum];
    int v3_stss[v3_maxnum];
    unsigned int v3_num;
    int sts, i;
    char *diskunits = getenv("PCP_DISK_UNITS");
    struct timeval t;

    sts = pmCreateFetchGroup(&fg, PM_CONTEXT_HOST, "local:");
    pcpassert(sts);
    sts = pmExtendFetchGroup_item(fg, "kernel.all.load", "1 minute",
                                  NULL, &v, PM_TYPE_FLOAT, NULL);
    pcpassert(sts);
    sts = pmExtendFetchGroup_item(fg, "kernel.all.idletime", NULL,
                                  "hour", &v2, PM_TYPE_DOUBLE, NULL);
    pcpassert(sts);
    sts = pmExtendFetchGroup_indom(fg, "disk.dev.total", diskunits,
                                   NULL, v3_names,
                                   v3_values, PM_TYPE_STRING,
                                   v3_stss, v3_maxnum, &v3_num, NULL);
    pcpassert(sts);
    sts = pmExtendFetchGroup_timestamp(fg, &t);
    pcpassert(sts);

    for (i=0; i < 10; i++) {
        unsigned int j;
        char stamp[28];

        sts = pmFetchGroup(fg);
        pcpassert(sts);
        printf("%s", pmCtime(&t.tv_sec, stamp));
        printf("1-minute load: %f; idletime: %f h\\n", v.f, v2.d);
        for (j=0; j < v3_num; j++) {
            if (v3_stss[j] == 0)
                 printf("disk %s i/o operations (%s): %s\\n",
                        v3_names[j] ? v3_names[j] : "?",
                        diskunits ? diskunits : "-",
                        v3_values[j].cp);
        }
        sleep(1);
    }

    sts = pmDestroyFetchGroup(fg);
    pcpassert(sts);
    return 0;
}
.fi
.SH "UNUSUAL SITUATIONS"
The fetchgroup API supports only the numeric, string and event metric
types.
Aggregates are rejected during
\fBpmExtendFetchGroup_*\fP.
.PP
Any strings supplied by the fetchgroup API to the application are
"owned" by the API.
The application should consider them read-only,
so it should not modify them nor
.B free
them.
.PP
Error codes are always negative integers, whether returned from
fetchgroup functions as return value, or stored in \fIout_sts\fP
type variables.
Normal result codes are always zero.
.PP
Because of the unique ways in which extracted data is shared between
the application and a fetchgroup, the functions in this API are \fInot
protected\fP by the multi-threading mutexes conventional in other
parts of PMAPI.
Specifically, for any given \fBpmFG\fP, it is \fInot
safe\fP to concurrently call two or more fetchgroup API functions, nor
to traverse the registered output variables while calling one of the
functions.
Instead, the calling application must ensure that only one
thread at a time uses these calls \fIand\fP the registered output
variables.
On the other hand, concurrency between different
\fBpmFG\fP instances is unrestricted, because they share no global
data.
.PP
Any pointers passed to a successful \fBpmFetchGroupExtent_*\fP call
must stay valid throughout the lifetime of the fetchgroup, since
future \fBpmFetchGroup\fP calls may write into them.
.SH DIAGNOSTICS
The fetchgroup API offers several options for collecting diagnostics.
Negative integer error codes may be returned from each function for
serious conditions.
.PP
In addition, each output pmAtomValue may have a corresponding integer
variable, where \fBpmFetchGroup\fP can store per-metric per-instance
error codes.
.PP
As an alternative, per-metric per-instance error conditions are also
signalled by setting the corresponding pmAtomValue to a sentinel
value.
If unambiguous and precise error detection is not required, this
may be sufficient.
The sentinel value is negative one for all integers (including unsigned
integers \- i.e. all bits are set), \fBNaN\fP
for floating point types, a NULL pointer for strings, and 0.0s for the
timestamp.
The fetchgroup API guarantees that once an output
pmAtomValue is registered (during a successful
\fBpmExtendFetchGroup_*\fP call), it will be cleared to the sentinel
value or to a valid converted metric value, from the time of
registration until the \fBpmDestroyFetchGroup\fP call.
.SH "SEE ALSO"
.BR PMAPI (3),
.BR pmLookupName (3),
.BR pmFetchHighRes (3),
.BR pmParseUnitsStr (3),
.BR pmUseContext (3),
.BR pmRegisterDerived (3)
and
.BR pmExtractValue (3).