NAME¶

SYNOPSIS¶

DESCRIPTION¶

•

When compressing, the suffix of the target file format (.xz or .lzma) is appended to the source filename to get the target filename.

•

When decompressing, the .xz or .lzma suffix is removed from the filename to get the target filename. xz also recognizes the suffixes .txz and .tlz, and replaces them with the .tar suffix.

•

File is not a regular file. Symbolic links are not followed, and thus they are not considered to be regular files.

•

File has more than one hard link.

•

File has setuid, setgid, or sticky bit set.

•

The operation mode is set to compress and the file already has a suffix of the target file format (.xz or .txz when compressing to the .xz format, and .lzma or .tlz when compressing to the .lzma format).

•

The operation mode is set to decompress and the file doesn't have a suffix of any of the supported file formats (.xz, .txz, .lzma, or .tlz).

Memory usage¶

Concatenation and padding with .xz files¶

OPTIONS¶

Integer suffixes and special values¶

KiB

Multiply the integer by 1,024 (2^10). Ki, k, kB, K, and KB are accepted as synonyms for KiB.

MiB

Multiply the integer by 1,048,576 (2^20). Mi, m, M, and MB are accepted as synonyms for MiB.

GiB

Multiply the integer by 1,073,741,824 (2^30). Gi, g, G, and GB are accepted as synonyms for GiB.

Operation mode¶

-z, --compress

Compress. This is the default operation mode when no operation mode option is specified and no other operation mode is implied from the command name (for example, unxz implies --decompress).

-d, --decompress, --uncompress

Decompress.

-t, --test

Test the integrity of compressed files. This option is equivalent to --decompress --stdout except that the decompressed data is discarded instead of being written to standard output. No files are created or removed.

-l, --list

Print information about compressed files. No uncompressed output is produced, and no files are created or removed. In list mode, the program cannot read the compressed data from standard input or from other unseekable sources.

The default listing shows basic information about files, one file per line. To get more detailed information, use also the --verbose option. For even more information, use --verbose twice, but note that this may be slow, because getting all the extra information requires many seeks. The width of verbose output exceeds 80 characters, so piping the output to e.g. less -S may be convenient if the terminal isn't wide enough.

The exact output may vary between xz versions and different locales. For machine-readable output, --robot --list should be used.

Operation modifiers¶

-k, --keep

Don't delete the input files.

-f, --force

This option has several effects:

-c, --stdout, --to-stdout

Write the compressed or decompressed data to standard output instead of a file. This implies --keep.

--single-stream

Decompress only the first .xz stream, and silently ignore possible remaining input data following the stream. Normally such trailing garbage makes xz display an error.

xz never decompresses more than one stream from .lzma files or raw streams, but this option still makes xz ignore the possible trailing data after the .lzma file or raw stream.

This option has no effect if the operation mode is not --decompress or --test.

--no-sparse

Disable creation of sparse files. By default, if decompressing into a regular file, xz tries to make the file sparse if the decompressed data contains long sequences of binary zeros. It also works when writing to standard output as long as standard output is connected to a regular file and certain additional conditions are met to make it safe. Creating sparse files may save disk space and speed up the decompression by reducing the amount of disk I/O.

-S .suf, --suffix=.suf

When compressing, use .suf as the suffix for the target file instead of .xz or .lzma. If not writing to standard output and the source file already has the suffix .suf, a warning is displayed and the file is skipped.

When decompressing, recognize files with the suffix .suf in addition to files with the .xz, .txz, .lzma, or .tlz suffix. If the source file has the suffix .suf, the suffix is removed to get the target filename.

When compressing or decompressing raw streams (--format=raw), the suffix must always be specified unless writing to standard output, because there is no default suffix for raw streams.

--files[=file]

Read the filenames to process from file; if file is omitted, filenames are read from standard input. Filenames must be terminated with the newline character. A dash (-) is taken as a regular filename; it doesn't mean standard input. If filenames are given also as command line arguments, they are processed before the filenames read from file.

--files0[=file]

This is identical to --files[=file] except that each filename must be terminated with the null character.

Basic file format and compression options¶

-F format, --format=format

Specify the file format to compress or decompress:

-C check, --check=check

Specify the type of the integrity check. The check is calculated from the uncompressed data and stored in the .xz file. This option has an effect only when compressing into the .xz format; the .lzma format doesn't support integrity checks. The integrity check (if any) is verified when the .xz file is decompressed.

Supported check types:

Integrity of the .xz headers is always verified with CRC32. It is not possible to change or disable it.

-0 ... -9

Select a compression preset level. The default is -6. If multiple preset levels are specified, the last one takes effect. If a custom filter chain was already specified, setting a compression preset level clears the custom filter chain.

The differences between the presets are more significant than with gzip(1) and bzip2(1). The selected compression settings determine the memory requirements of the decompressor, thus using a too high preset level might make it painful to decompress the file on an old system with little RAM. Specifically, it's not a good idea to blindly use -9 for everything like it often is with gzip(1) and bzip2(1).

On the same hardware, the decompression speed is approximately a constant number of bytes of compressed data per second. In other words, the better the compression, the faster the decompression will usually be. This also means that the amount of uncompressed output produced per second can vary a lot.

The following table summarises the features of the presets:

Column descriptions:

-e, --extreme

Use a slower variant of the selected compression preset level (-0 ... -9) to hopefully get a little bit better compression ratio, but with bad luck this can also make it worse. Decompressor memory usage is not affected, but compressor memory usage increases a little at preset levels -0 ... -3.

Since there are two presets with dictionary sizes 4 MiB and 8 MiB, the presets -3e and -5e use slightly faster settings (lower CompCPU) than -4e and -6e, respectively. That way no two presets are identical.

For example, there are a total of four presets that use 8 MiB dictionary, whose order from the fastest to the slowest is -5, -6, -5e, and -6e.

--fast

--best

These are somewhat misleading aliases for -0 and -9, respectively. These are provided only for backwards compatibility with LZMA Utils. Avoid using these options.

--block-size=size

When compressing to the .xz format, split the input data into blocks of size bytes. The blocks are compressed independently from each other.

--memlimit-compress=limit

Set a memory usage limit for compression. If this option is specified multiple times, the last one takes effect.

If the compression settings exceed the limit, xz will adjust the settings downwards so that the limit is no longer exceeded and display a notice that automatic adjustment was done. Such adjustments are not made when compressing with --format=raw or if --no-adjust has been specified. In those cases, an error is displayed and xz will exit with exit status 1.

The limit can be specified in multiple ways:

See also the section Memory usage.

--memlimit-decompress=limit

Set a memory usage limit for decompression. This also affects the --list mode. If the operation is not possible without exceeding the limit, xz will display an error and decompressing the file will fail. See --memlimit-compress=limit for possible ways to specify the limit.

-M limit, --memlimit=limit, --memory=limit

This is equivalent to specifying --memlimit-compress=limit --memlimit-decompress=limit.

--no-adjust

Display an error and exit if the compression settings exceed the memory usage limit. The default is to adjust the settings downwards so that the memory usage limit is not exceeded. Automatic adjusting is always disabled when creating raw streams (--format=raw).

-T threads, --threads=threads

Specify the number of worker threads to use. The actual number of threads can be less than threads if using more threads would exceed the memory usage limit.

Multithreaded compression and decompression are not implemented yet, so this option has no effect for now.

As of writing (2010-09-27), it hasn't been decided if threads will be used by default on multicore systems once support for threading has been implemented. Comments are welcome. The complicating factor is that using many threads will increase the memory usage dramatically. Note that if multithreading will be the default, it will probably be done so that single-threaded and multithreaded modes produce the same output, so compression ratio won't be significantly affected if threading will be enabled by default.

Custom compressor filter chains¶

--lzma1[=options]

--lzma2[=options]

Add LZMA1 or LZMA2 filter to the filter chain. These filters can be used only as the last filter in the chain.

LZMA1 is a legacy filter, which is supported almost solely due to the legacy .lzma file format, which supports only LZMA1. LZMA2 is an updated version of LZMA1 to fix some practical issues of LZMA1. The .xz format uses LZMA2 and doesn't support LZMA1 at all. Compression speed and ratios of LZMA1 and LZMA2 are practically the same.

LZMA1 and LZMA2 share the same set of options:

When decoding raw streams (--format=raw), LZMA2 needs only the dictionary size. LZMA1 needs also lc, lp, and pb.

--x86[=options]

--powerpc[=options]

--ia64[=options]

--arm[=options]

--armthumb[=options]

--sparc[=options]

Add a branch/call/jump (BCJ) filter to the filter chain. These filters can be used only as a non-last filter in the filter chain.

A BCJ filter converts relative addresses in the machine code to their absolute counterparts. This doesn't change the size of the data, but it increases redundancy, which can help LZMA2 to produce 0-15 % smaller .xz file. The BCJ filters are always reversible, so using a BCJ filter for wrong type of data doesn't cause any data loss, although it may make the compression ratio slightly worse.

It is fine to apply a BCJ filter on a whole executable; there's no need to apply it only on the executable section. Applying a BCJ filter on an archive that contains both executable and non-executable files may or may not give good results, so it generally isn't good to blindly apply a BCJ filter when compressing binary packages for distribution.

These BCJ filters are very fast and use insignificant amount of memory. If a BCJ filter improves compression ratio of a file, it can improve decompression speed at the same time. This is because, on the same hardware, the decompression speed of LZMA2 is roughly a fixed number of bytes of compressed data per second.

These BCJ filters have known problems related to the compression ratio:

Both of the above problems will be fixed in the future in a new filter. The old BCJ filters will still be useful in embedded systems, because the decoder of the new filter will be bigger and use more memory.

Different instruction sets have have different alignment:

Since the BCJ-filtered data is usually compressed with LZMA2, the compression ratio may be improved slightly if the LZMA2 options are set to match the alignment of the selected BCJ filter. For example, with the IA-64 filter, it's good to set pb=4 with LZMA2 (2^4=16). The x86 filter is an exception; it's usually good to stick to LZMA2's default four-byte alignment when compressing x86 executables.

All BCJ filters support the same options:

--delta[=options]

Add the Delta filter to the filter chain. The Delta filter can be only used as a non-last filter in the filter chain.

Currently only simple byte-wise delta calculation is supported. It can be useful when compressing e.g. uncompressed bitmap images or uncompressed PCM audio. However, special purpose algorithms may give significantly better results than Delta + LZMA2. This is true especially with audio, which compresses faster and better e.g. with flac(1).

Supported options:

Other options¶

-q, --quiet

Suppress warnings and notices. Specify this twice to suppress errors too. This option has no effect on the exit status. That is, even if a warning was suppressed, the exit status to indicate a warning is still used.

-v, --verbose

Be verbose. If standard error is connected to a terminal, xz will display a progress indicator. Specifying --verbose twice will give even more verbose output.

The progress indicator shows the following information:

When standard error is not a terminal, --verbose will make xz print the filename, compressed size, uncompressed size, compression ratio, and possibly also the speed and elapsed time on a single line to standard error after compressing or decompressing the file. The speed and elapsed time are included only when the operation took at least a few seconds. If the operation didn't finish, e.g. due to user interruption, also the completion percentage is printed if the size of the input file is known.

-Q, --no-warn

Don't set the exit status to 2 even if a condition worth a warning was detected. This option doesn't affect the verbosity level, thus both --quiet and --no-warn have to be used to not display warnings and to not alter the exit status.

--robot

Print messages in a machine-parsable format. This is intended to ease writing frontends that want to use xz instead of liblzma, which may be the case with various scripts. The output with this option enabled is meant to be stable across xz releases. See the section ROBOT MODE for details.

--info-memory

Display, in human-readable format, how much physical memory (RAM) xz thinks the system has and the memory usage limits for compression and decompression, and exit successfully.

-h, --help

Display a help message describing the most commonly used options, and exit successfully.

-H, --long-help

Display a help message describing all features of xz, and exit successfully

-V, --version

Display the version number of xz and liblzma in human readable format. To get machine-parsable output, specify --robot before --version.

ROBOT MODE¶

Version¶

X

Major version.

YYY

Minor version. Even numbers are stable. Odd numbers are alpha or beta versions.

ZZZ

Patch level for stable releases or just a counter for development releases.

S

Stability. 0 is alpha, 1 is beta, and 2 is stable. S should be always 2 when YYY is even.

Memory limit information¶

1.

Total amount of physical memory (RAM) in bytes

2.

Memory usage limit for compression in bytes. A special value of zero indicates the default setting, which for single-threaded mode is the same as no limit.

3.

Memory usage limit for decompression in bytes. A special value of zero indicates the default setting, which for single-threaded mode is the same as no limit.

List mode¶

name

This is always the first line when starting to list a file. The second column on the line is the filename.

file

This line contains overall information about the .xz file. This line is always printed after the name line.

stream

This line type is used only when --verbose was specified. There are as many stream lines as there are streams in the .xz file.

block

This line type is used only when --verbose was specified. There are as many block lines as there are blocks in the .xz file. The block lines are shown after all the stream lines; different line types are not interleaved.

summary

This line type is used only when --verbose was specified twice. This line is printed after all block lines. Like the file line, the summary line contains overall information about the .xz file.

totals

This line is always the very last line of the list output. It shows the total counts and sizes.

EXIT STATUS¶

0

All is good.

1

An error occurred.

2

Something worth a warning occurred, but no actual errors occurred.

ENVIRONMENT¶

XZ_DEFAULTS

User-specific or system-wide default options. Typically this is set in a shell initialization script to enable xz's memory usage limiter by default. Excluding shell initialization scripts and similar special cases, scripts must never set or unset XZ_DEFAULTS.

XZ_OPT

This is for passing options to xz when it is not possible to set the options directly on the xz command line. This is the case e.g. when xz is run by a script or tool, e.g. GNU tar(1):

XZ_OPT=-2v tar caf foo.tar.xz foo

Scripts may use XZ_OPT e.g. to set script-specific default compression options. It is still recommended to allow users to override XZ_OPT if that is reasonable, e.g. in sh(1) scripts one may use something like this:

XZ_OPT=${XZ_OPT-"-7e"}
export XZ_OPT

LZMA UTILS COMPATIBILITY¶

Compression preset levels¶

Streamed vs. non-streamed .lzma files¶

Unsupported .lzma files¶

Trailing garbage¶

NOTES¶

Compressed output may vary¶

Embedded .xz decompressors¶

EXAMPLES¶

Basics¶

xz foo

xz -dk bar.xz

tar cf - baz | xz -4e > baz.tar.xz

xz -dcf a.txt b.txt.xz c.txt d.txt.lzma > abcd.txt

Parallel compression of many files¶

find . -type f \! -name '*.xz' -print0 \
    | xargs -0r -P4 -n16 xz -T1

Robot mode¶

xz --robot --list *.xz | awk '/^totals/{print $5-$4}'

if ! eval "$(xz --robot --version 2> /dev/null)" ||
        [ "$XZ_VERSION" -lt 50000002 ]; then
    echo "Your xz is too old."
fi
unset XZ_VERSION LIBLZMA_VERSION

NEWLIM=$((123 << 20))  # 123 MiB
OLDLIM=$(xz --robot --info-memory | cut -f3)
if [ $OLDLIM -eq 0 -o $OLDLIM -gt $NEWLIM ]; then
    XZ_OPT="$XZ_OPT --memlimit-decompress=$NEWLIM"
    export XZ_OPT
fi

Custom compressor filter chains¶

xz --lzma2=preset=1,dict=32MiB foo.tar

xz -vv --lzma2=dict=192MiB big_foo.tar

xz --check=crc32 --lzma2=preset=6e,dict=64KiB foo

xz --lzma2=preset=6e,pb=0,lc=4 source_code.tar

xz --x86 --lzma2 libfoo.so

xz --delta=dist=3 --lzma2=pb=0 foo.tiff

SEE ALSO¶