table of contents
- bookworm 6.1.106-3
- bookworm-backports 6.10.11-1~bpo12+1
- testing 6.10.11-1
- unstable 6.11.2-1
- experimental 6.11-1~exp1
PERF-REPORT(1) | perf Manual | PERF-REPORT(1) |
NAME¶
perf-report - Read perf.data (created by perf record) and display the profile
SYNOPSIS¶
perf report [-i <file> | --input=file]
DESCRIPTION¶
This command displays the performance counter profile information recorded via perf record.
OPTIONS¶
-i, --input=
-v, --verbose
-q, --quiet
-n, --show-nr-samples
--show-cpu-utilization
-T, --threads
-c, --comms=
--pid=
--tid=
-d, --dsos=
-S, --symbols=
--symbol-filter=
-U, --hide-unresolved
-s, --sort=
Each key has following meaning:
By default, comm, dso and symbol keys are used. (i.e. --sort comm,dso,symbol)
If --branch-stack option is used, following sort keys are also available:
And default sort keys are changed to comm, dso_from, symbol_from, dso_to and symbol_to, see '--branch-stack'.
When the sort key symbol is specified, columns "IPC" and "IPC Coverage" are enabled automatically. Column "IPC" reports the average IPC per function and column "IPC coverage" reports the percentage of instructions with sampled IPC in this function. IPC means Instruction Per Cycle. If it's low, it indicates there may be a performance bottleneck when the function is executed, such as a memory access bottleneck. If a function has high overhead and low IPC, it's worth further analyzing it to optimize its performance.
If the --mem-mode option is used, the following sort keys are also available (incompatible with --branch-stack): symbol_daddr, dso_daddr, locked, tlb, mem, snoop, dcacheline, blocked.
And the default sort keys are changed to local_weight, mem, sym, dso, symbol_daddr, dso_daddr, snoop, tlb, locked, blocked, local_ins_lat, see '--mem-mode'.
If the data file has tracepoint event(s), following (dynamic) sort keys are also available: trace, trace_fields, [<event>.]<field>[/raw]
The last form consists of event and field names. If event name is omitted, it searches all events for matching field name. The matched field will be shown only for the event has the field. The event name supports substring match so user doesn't need to specify full subsystem and event name everytime. For example, 'sched:sched_switch' event can be shortened to 'switch' as long as it's not ambiguous. Also event can be specified by its index (starting from 1) preceded by the '%'. So '%1' is the first event, '%2' is the second, and so on.
The field name can have '/raw' suffix which disables pretty printing and shows raw field value like hex numbers. The --raw-trace option has the same effect for all dynamic sort keys.
The default sort keys are changed to 'trace' if all events in the data file are tracepoint.
-F, --fields=
Also it can contain any sort key(s).
By default, every sort keys not specified in -F will be appended automatically.
If the keys starts with a prefix '+', then it will append the specified
field(s) to the default field order. For example: perf report -F +period,sample.
-p, --parent=<regex>
A regex filter to identify parent. The parent is a caller of this function and searched through the callchain, thus it requires callchain information recorded. The pattern is in the extended regex format and defaults to "^sys_|^do_page_fault", see --sort parent.
-x, --exclude-other
-w, --column-widths=<width[,width...]>
-t, --field-separator=
-D, --dump-raw-trace
--disable-order
-g, --call-graph=<print_type,threshold[,print_limit],order,sort_key[,branch],value>
Display call chains using type, min percent threshold, print limit, call order, sort key, optional branch and value. Note that ordering is not fixed so any parameter can be given in an arbitrary order. One exception is the print_limit which should be preceded by threshold.
print_type can be either: - flat: single column, linear exposure of call chains. - graph: use a graph tree, displaying absolute overhead rates. (default) - fractal: like graph, but displays relative rates. Each branch of
the tree is considered as a new profiled object. - folded: call chains are displayed in a line, separated by semicolons - none: disable call chain display.
threshold is a percentage value which specifies a minimum percent to be included in the output call graph. Default is 0.5 (%).
print_limit is only applied when stdio interface is used. It's to limit number of call graph entries in a single hist entry. Note that it needs to be given after threshold (but not necessarily consecutive). Default is 0 (unlimited).
order can be either: - callee: callee based call graph. - caller: inverted caller based call graph. Default is 'caller' when --children is used, otherwise 'callee'.
sort_key can be: - function: compare on functions (default) - address: compare on individual code addresses - srcline: compare on source filename and line number
branch can be: - branch: include last branch information in callgraph when available.
Usually more convenient to use --branch-history for this.
value can be: - percent: display overhead percent (default) - period: display event period - count: display event count
--children
--max-stack
Default: 127
-G, --inverted
--ignore-callees=<regex>
--pretty=<key>
--stdio
--stdio-color
--tui
--gtk
-k, --vmlinux=<file>
--ignore-vmlinux
--kallsyms=<file>
-m, --modules
-f, --force
--symfs=<directory>
-C, --cpu
-M, --disassembler-style=
--source
--asm-raw
--show-total-period
-I, --show-info
-b, --branch-stack
--branch-history
--addr2line=<path>
--objdump=<path>
--prefix=PREFIX, --prefix-strip=N
--group
--group-sort-idx
--demangle
--demangle-kernel
--mem-mode
--percent-limit
--percentage
"relative" means it's relative to filtered entries only so that the sum of shown entries will be always 100%. "absolute" means it retains the original value before and after the filter is applied.
--header
--header-only
--time
Also support time percent with multiple time ranges. Time string is 'a%/n,b%/m,...' or 'a%-b%,c%-%d,...'.
For example: Select the second 10% time slice:
perf report --time 10%/2
Select from 0% to 10% time slice:
perf report --time 0%-10%
Select the first and second 10% time slices:
perf report --time 10%/1,10%/2
Select from 0% to 10% and 30% to 40% slices:
perf report --time 0%-10%,30%-40%
--switch-on EVENT_NAME
This may be interesting to measure a workload only after some initialization phase is over, i.e. insert a perf probe at that point and then using this option with that probe.
--switch-off EVENT_NAME
--show-on-off-events
--itrace
i synthesize instructions events y synthesize cycles events b synthesize branches events (branch misses for Arm SPE) c synthesize branches events (calls only) r synthesize branches events (returns only) x synthesize transactions events w synthesize ptwrite events p synthesize power events (incl. PSB events for Intel PT) o synthesize other events recorded due to the use
of aux-output (refer to perf record) I synthesize interrupt or similar (asynchronous) events
(e.g. Intel PT Event Trace) e synthesize error events d create a debug log f synthesize first level cache events m synthesize last level cache events M synthesize memory events t synthesize TLB events a synthesize remote access events g synthesize a call chain (use with i or x) G synthesize a call chain on existing event records l synthesize last branch entries (use with i or x) L synthesize last branch entries on existing event records s skip initial number of events q quicker (less detailed) decoding A approximate IPC Z prefer to ignore timestamps (so-called "timeless" decoding) T use the timestamp trace as kernel time
The default is all events i.e. the same as --itrace=iybxwpe, except for perf script where it is --itrace=ce
In addition, the period (default 100000, except for perf script where it is 1) for instructions events can be specified in units of:
i instructions t ticks ms milliseconds us microseconds ns nanoseconds (default)
Also the call chain size (default 16, max. 1024) for instructions or transactions events can be specified.
Also the number of last branch entries (default 64, max. 1024) for instructions or transactions events can be specified.
Similar to options g and l, size may also be specified for options G and L. On x86, note that G and L work poorly when data has been recorded with large PEBS. Refer linkperf:perf-intel-pt[1] man page for details.
It is also possible to skip events generated (instructions, branches, transactions, ptwrite, power) at the beginning. This is useful to ignore initialization code.
--itrace=i0nss1000000
skips the first million instructions.
The 'e' option may be followed by flags which affect what errors will or will not be reported. Each flag must be preceded by either '+' or '-'. The flags are:
o overflow
l trace data lost
If supported, the 'd' option may be followed by flags which affect what debug messages will or will not be logged. Each flag must be preceded by either '+' or '-'. The flags are:
a all perf events
e output only on errors (size configurable - see linkperf:perf-config[1])
o output to stdout
If supported, the 'q' option may be repeated to increase the effect.
To disable decoding entirely, use --no-itrace.
--full-source-path
--show-ref-call-graph
--stitch-lbr
--socket-filter
--samples=N
--raw-trace
-H, --hierarchy
For example: In normal output:
perf report -s dso,sym # Overhead Shared Object Symbol
50.00% [kernel.kallsyms] [k] kfunc1
20.00% perf [.] foo
15.00% [kernel.kallsyms] [k] kfunc2
10.00% perf [.] bar
5.00% libc.so [.] libcall
In hierarchy output:
perf report -s dso,sym --hierarchy # Overhead Shared Object / Symbol
65.00% [kernel.kallsyms]
50.00% [k] kfunc1
15.00% [k] kfunc2
30.00% perf
20.00% [.] foo
10.00% [.] bar
5.00% libc.so
5.00% [.] libcall
--inline
--mmaps
Please note that not all mmaps are stored, options affecting which ones are include 'perf record --data', for instance.
--ns
--stats
--tasks
--percent-type
The local/global keywords set if the percentage is computed in the scope of the function (local) or the whole data (global). The period/hits keywords set the base the percentage is computed on - the samples period or the number of samples (hits).
--time-quantum
--total-cycles
'Sampled Cycles%' - block sampled cycles aggregation / total sampled cycles 'Sampled Cycles' - block sampled cycles aggregation 'Avg Cycles%' - block average sampled cycles / sum of total block average
sampled cycles 'Avg Cycles' - block average sampled cycles
--skip-empty
OVERHEAD CALCULATION¶
The overhead can be shown in two columns as Children and Self when perf collects callchains. The self overhead is simply calculated by adding all period values of the entry - usually a function (symbol). This is the value that perf shows traditionally and sum of all the self overhead values should be 100%.
The children overhead is calculated by adding all period values of the child functions so that it can show the total overhead of the higher level functions even if they don’t directly execute much. Children here means functions that are called from another (parent) function.
It might be confusing that the sum of all the children overhead values exceeds 100% since each of them is already an accumulation of self overhead of its child functions. But with this enabled, users can find which function has the most overhead even if samples are spread over the children.
Consider the following example; there are three functions like below.
void foo(void) {
/* do something */ } void bar(void) {
/* do something */
foo(); } int main(void) {
bar()
return 0; }
In this case foo is a child of bar, and bar is an immediate child of main so foo also is a child of main. In other words, main is a parent of foo and bar, and bar is a parent of foo.
Suppose all samples are recorded in foo and bar only. When it’s recorded with callchains the output will show something like below in the usual (self-overhead-only) output of perf report:
Overhead Symbol ........ .....................
60.00% foo
|
--- foo
bar
main
__libc_start_main
40.00% bar
|
--- bar
main
__libc_start_main
When the --children option is enabled, the self overhead values of child functions (i.e. foo and bar) are added to the parents to calculate the children overhead. In this case the report could be displayed as:
Children Self Symbol ........ ........ ....................
100.00% 0.00% __libc_start_main
|
--- __libc_start_main
100.00% 0.00% main
|
--- main
__libc_start_main
100.00% 40.00% bar
|
--- bar
main
__libc_start_main
60.00% 60.00% foo
|
--- foo
bar
main
__libc_start_main
In the above output, the self overhead of foo (60%) was add to the children overhead of bar, main and _\_libc_start_main. Likewise, the self overhead of bar (40%) was added to the children overhead of main and \_\_libc_start_main.
So \_\_libc_start_main and main are shown first since they have same (100%) children overhead (even though they have zero self overhead) and they are the parents of foo and bar.
Since v3.16 the children overhead is shown by default and the output is sorted by its values. The children overhead is disabled by specifying --no-children option on the command line or by adding report.children = false or top.children = false in the perf config file.
SEE ALSO¶
perf-stat(1), perf-annotate(1), perf-record(1), perf-intel-pt(1)
2024-10-03 | perf |