NAME¶

SYNOPSIS¶

DESCRIPTION¶

OPTIONS¶

-b SIZE, --buffer=SIZE

Size of internal buffer in which trace data will be saved. Default size is 128k.

--daemon

(XXX: rename to dont-wait or keep) Trace a daemon process which calls fork(2) and then exit(2). Usually uftrace stops recording when its child has exited, but a daemon processes calls exit(2) before doing its real job (in the forked child process). This option is used to keep tracing such daemon processes.

-F FUNC, --filter=FUNC

Set filter to trace selected functions only. This option can be used more than once. See FILTERS.

-N FUNC, --notrace=FUNC

Set filter not to trace selected functions (or the functions called underneath them). This option can be used more than once. See FILTERS.

-T TRG, --trigger=TRG

Set trigger on selected functions. This option can be used more than once. See TRIGGERS.

-D DEPTH, --depth=DEPTH

Set global trace limit in nesting level.

-t TIME, --time-filter=TIME

Do not show small functions under the time threshold. If some functions explicitly have 'trace' trigger, those are always traced regardless of execution time.

-A SPEC, --argument=SPEC

Record function arguments. This option can be used more than once. See ARGUMENTS.

-R SPEC, --retval=SPEC

Record function return value. This option can be used more than once. See ARGUMENTS.

--force

Allow running uftrace even if some problems occur. When uftrace record finds no mcount symbol (which is generated by compiler) in the executable, it quits with an error message since uftrace can not trace the executable. However, it is possible that the user is only interested in functions within a dynamically-linked library, in which case this option can be used to cause uftrace to run the program regardless. Also, the -A/--argument and -R/--retval options work only for binaries built with -pg, so uftrace will normally exit when it tries to run binaries built without that option. This option ignores the warning and goes on tracing without the argument and/or return value.

--flat

Print flat format rather than C-like format. This is usually for debugging and testing purpose.

-L PATH, --library-path=PATH

Load necessary internal libraries from this path. This is for testing.

-k, --kernel

Trace kernel functions as well as user functions. This is simply a shortcut to --kernel-depth=1.

--no-libcall

Do not record library function invocations. Library calls are normally traced by hooking the dynamic linker's resolve function in the PLT. One can disable it with this option.

--no-pltbind

Do not bind dynamic symbol address. This option uses the LD_BIND_NOT environment variable to trace library function calls which might be missing due to concurrent (first) accesses. It is not meaningful to use this option with the --no-libcall option.

--disable

Start uftrace with tracing disabled. This is only meaningful when used with a trace_on trigger.

--demangle=TYPE

Demangle C++ symbol names. Possible values are "full", "simple" and "no". Default is "simple" which ignores function arguments and template parameters.

--report

Show live-report before replay.

--column-view

Show each task in separate column. This makes easy to distinguish functions in different tasks.

--column-offset=DEPTH

When --column-view option is used, this option specifies the amount of offset between each task. Default is 8.

--task-newline

Interleave a new line when task is changed. This makes easy to distinguish functions in different tasks.

--num-thread=NUM

Use NUM threads to record trace data. Default is 1/4 of online CPUs (but when full kernel tracing is enabled, it will use the full number of CPUs).

--no-comment

Do not show comments of returned functions.

--libmcount-single

Use single thread version of libmcount for faster recording. This is ignored if the target program calls pthread_create().

--rt-prio=PRIO

Boost priority of recording threads to real-time (FIFO) with priority of PRIO. This is particularly useful for high-volume data such as full kernel tracing.

-K DEPTH, --kernel-depth=DEPTH

Set kernel max function depth separately. Implies --kernel.

--kernel-buffer=SIZE

Set kernel tracing buffer size. The default value (in the kernel) is 1408k. Implies --kernel.

--kernel-skip-out

Do not show kernel functions called outside of user functions. This option is deprecated and set to true by default.

--kernel-full

Show all kernel functions called outside of user functions. This option is the inverse of --kernel-skip-out. Implies --kernel.

FILTERS¶

The first one is an opt-in filter. By default, it doesn't trace anything. But when one of the specified functions is executed, tracing is started. When the function returns, tracing is stopped again.

For example, consider a simple program which calls a(), b() and c() in turn.

$ cat abc.c
void c(void) {
    /* do nothing */
}
void b(void) {
    c();
}
void a(void) {
    b();
}
int main(void) {
    a();
    return 0;
}
$ gcc -o abc abc.c

    

Normally uftrace will trace all the functions from main() to c().

$ uftrace ./abc
# DURATION    TID     FUNCTION
 138.494 us [ 1234] | __cxa_atexit();
            [ 1234] | main() {
            [ 1234] |   a() {
            [ 1234] |     b() {
   3.880 us [ 1234] |       c();
   5.475 us [ 1234] |     } /* b */
   6.448 us [ 1234] |   } /* a */
   8.631 us [ 1234] | } /* main */

    

But when the -F b filter option is used, it will not trace main() or a() but only b() and c().

$ uftrace -F b ./abc
# DURATION    TID     FUNCTION
            [ 1234] | b() {
   3.880 us [ 1234] |   c();
   5.475 us [ 1234] | } /* b */

    

The second type of filter is opt-out. By default, everything is traced, but when one of the specified functions is executed, tracing stops. When the excluded function returns, tracing is started again.

In the above example, you can omit the function b() and all calls it makes with the -N option.

$ uftrace live -N b ./abc
# DURATION    TID     FUNCTION
 138.494 us [ 1234] | __cxa_atexit();
            [ 1234] | main() {
   6.448 us [ 1234] |   a();
   8.631 us [ 1234] | } /* main */

    

In addition, you can limit the print nesting level with the -D option.

$ uftrace -D 3 ./abc
# DURATION    TID     FUNCTION
 138.494 us [ 1234] | __cxa_atexit();
            [ 1234] | main() {
            [ 1234] |   a() {
   5.475 us [ 1234] |     b();
   6.448 us [ 1234] |   } /* a */
   8.631 us [ 1234] | } /* main */

    

In the above example, uftrace only prints functions up to a depth of 3, so leaf function c() was omitted. Note that the -D option works with -F.

Sometimes it's useful to see long-running functions only. This is good because there are usually many tiny functions that are not interesting. The -t/--time-filter option implements the time-based filter that only records functions which run longer than the given threshold. In the above example, the user might want to see functions running more than 5 microseconds like below:

$ uftrace live -t 5us ./abc
# DURATION    TID     FUNCTION
 138.494 us [ 1234] | __cxa_atexit();
            [ 1234] | main() {
            [ 1234] |   a() {
   5.475 us [ 1234] |     b();
   6.448 us [ 1234] |   } /* a */
   8.631 us [ 1234] | } /* main */

    

You can also set triggers on filtered functions. See TRIGGERS section below for details.

TRIGGERS¶

<trigger>  :=  <symbol> "@" <actions>
<actions>  :=  <action>  | <action> "," <actions>
<action>   :=  "depth="<num> | "backtrace" | "trace" | "trace_on" | "trace_off" | "recover" | "color"=<color>

    

The depth trigger is to change filter depth during execution of the function. It can be used to apply different filter depths for different functions. And the backtrace trigger is used to print a stack backtrace at replay time.

The color trigger is to change the color of the function in replay output. The supported colors are red, green, blue, yellow, magenta, cyan, bold, and gray.

The following example shows how triggers work. The global filter maximum depth is 5, but when function b() is called, it is changed to 1, so functions below b() will not shown.

$ uftrace live -D 5 -T 'b@depth=1' ./abc
# DURATION    TID     FUNCTION
 138.494 us [ 1234] | __cxa_atexit();
            [ 1234] | main() {
            [ 1234] |   a() {
   5.475 us [ 1234] |     b();
   6.448 us [ 1234] |   } /* a */
   8.631 us [ 1234] | } /* main */

    

The backtrace trigger is only meaningful in the replay command.

The traceon and traceoff actions (the _ can be omitted from trace_on and trace_off) control whether uftrace records the specified functions or not.

Triggers only work for user-level functions for now.

ARGUMENTS¶

<argument>    :=  <symbol> "@" <specs>
<specs>       :=  <spec> | <spec> "," <spec>
<spec>        :=  ( <int_spec> | <float_spec> | <ret_spec> )
<int_spec>    :=  "arg" N [ "/" <format> [ <size> ] ]
<float_spec>  :=  "fparg" N [ "/" ( <size> | "80" ) ]
<ret_spec>    :=  "retval" [ "/" <format> [ <size> ] ]
<format>      :=  "i" | "u" | "x" | "s" | "c" | "f"
<size>        :=  "8" | "16" | "32" | "64"

    

The -A/--argument option takes argN where N is an index of the arguments. The index starts from 1 and corresponds to the argument passing order of the calling convention on the system. Note that the indexes of arguments are separately counted for integer (or pointer) and floating-point type, and they can interfere depending on the calling convention. The argN is for integer arguments and fpargN is for floating-point arguments.

Users can optionally specify a format and size for the arguments and/or return values. Without this, uftrace treats them as 'long int' type for integers and 'double' for floating-point numbers. The "i" format makes it signed integer type and "u" format is for unsigned type. Both are printed as decimal while "x" format makes it printed as hexadecimal. The "s" format is for string type and "c" format is for character type. Finally, the "f" format is for floating-point type and is meaningful only for return value (generally). Note that fpargN doesn't take the format field since it's always floating-point.

Please beware when using string type arguments since it can crash the program if the (pointer) value is invalid.

It is also possible to specify a certain register name or stack offset for arguments (but not for return value). The following register names can be used for argument:

•

x86: rdi, rsi, rdx, rcx, r8, r9 (for integer), xmm[0-7] (for floating-point)

•

arm: r[0-3] (for integer), s[0-15] or d[0-7] (for floating-point)

Examples are below:

$ uftrace -A main@arg1/x -R main@retval/i32 ./abc
# DURATION    TID     FUNCTION
 138.494 us [ 1234] | __cxa_atexit();
            [ 1234] | main(0x1) {
            [ 1234] |   a() {
            [ 1234] |     b() {
   3.880 us [ 1234] |       c();
   5.475 us [ 1234] |     } /* b */
   6.448 us [ 1234] |   } /* a */
   8.631 us [ 1234] | } = 0; /* main */
$ uftrace -A puts@arg1/s -R puts@retval ./hello
Hello world
# DURATION    TID     FUNCTION
   1.457 us [21534] | __monstartup();
   0.997 us [21534] | __cxa_atexit();
            [21534] | main() {
   7.226 us [21534] |   puts("Hello world") = 12;
   8.708 us [21534] | } /* main */

    

Note that these arguments and return value are recorded only if the executable was built with the -pg option. Executables built with -finstrument-functions will cause uftrace to exit with an error message. Recording of arguments and return values only works with user-level functions for now.

SEE ALSO¶

AUTHORS¶