.\" Automatically generated by Podwrapper::Man 1.6.1 (Pod::Simple 3.40)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "nbdcopy 1"
.TH nbdcopy 1 "2021-02-09" "libnbd-1.6.1" "LIBNBD"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
nbdcopy \- copy to and from an NBD server
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
.Vb 6
\& nbdcopy [\-\-allocated] [\-C N|\-\-connections=N]
\&         [\-\-destination\-is\-zero|\-\-target\-is\-zero] [\-\-flush]
\&         [\-\-no\-extents] [\-p|\-\-progress|\-\-progress=FD]
\&         [\-R N|\-\-requests=N] [\-S N|\-\-sparse=N]
\&         [\-\-synchronous] [\-T N|\-\-threads=N]
\&         SOURCE DESTINATION
\&
\& SOURCE, DESTINATION := \- | FILE | DEVICE | NBD\-URI | [ CMD ARGS ... ]
\&
\& nbdcopy \-\-help
\&
\& nbdcopy \-V|\-\-version
.Ve
.SH "EXAMPLES"
.IX Header "EXAMPLES"
.SS "nbdcopy nbd://example.com local.img"
.IX Subsection "nbdcopy nbd://example.com local.img"
This copies everything from the \s-1NBD\s0 server at \f(CW\*(C`example.com\*(C'\fR to a
local file called \fIlocal.img\fR.
.SS "nbdcopy nbd://example.com \- | file \-"
.IX Subsection "nbdcopy nbd://example.com - | file -"
This streams the first part of the disk on the \s-1NBD\s0 server at
\&\f(CW\*(C`example.com\*(C'\fR into the \fBfile\fR\|(1) command.  Note here that \f(CW\*(C`\-\*(C'\fR means
to stream to stdout (and therefore into the pipe to the file command).
.PP
\&\fBnbdinfo\fR\|(1) is another way to detect the content on an \s-1NBD\s0 server.
.ie n .SS "nbdcopy \-p /dev/sdX ""nbd+unix:///?socket=/tmp/unixsock"""
.el .SS "nbdcopy \-p /dev/sdX ``nbd+unix:///?socket=/tmp/unixsock''"
.IX Subsection "nbdcopy -p /dev/sdX nbd+unix:///?socket=/tmp/unixsock"
Copy the full local hard disk \f(CW\*(C`/dev/sdX\*(C'\fR to the \s-1NBD\s0 server listening
on the Unix domain socket \fI/tmp/unixsock\fR.  Because of the \fI\-p\fR
option this will print a progress bar.
.SS "nbdcopy nbd://server1 nbd://server2"
.IX Subsection "nbdcopy nbd://server1 nbd://server2"
Copy a full disk from one \s-1NBD\s0 server to another.
.SS "nbdcopy \*(-- [ qemu-nbd \-f qcow2 disk.qcow ] disk.raw"
.IX Subsection "nbdcopy [ qemu-nbd -f qcow2 disk.qcow ] disk.raw"
Run \fBqemu\-nbd\fR\|(8) as a subprocess to open \fIdisk.qcow2\fR, which is
then copied to \fIdisk.raw\fR.  Note \f(CW\*(C`\-\-\*(C'\fR to prevent qemu-nbd flags from
being interpreted as nbdcopy flags.
.SS "cat disk1 disk2 | nbdcopy \*(-- \- [ qemu-nbd \-f qcow2 output.qcow2 ]"
.IX Subsection "cat disk1 disk2 | nbdcopy - [ qemu-nbd -f qcow2 output.qcow2 ]"
Concatenate two raw-format disk images into the qcow2 file
\&\fIoutput.qcow2\fR.  The output file has to be precreated.
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
nbdcopy copies to and from an \s-1NBD\s0 server.  It can upload a local file
to an \s-1NBD\s0 server, or download the contents of an \s-1NBD\s0 server to a local
file, device or stdin/stdout.  It can also copy between \s-1NBD\s0 servers.
.PP
The local file can be a file, a block device (eg. \f(CW\*(C`/dev/cdrom\*(C'\fR), or
\&\f(CW\*(C`\-\*(C'\fR which means stream in from stdin or stream out to stdout.
.PP
The \s-1NBD\s0 server can be specified using an \s-1NBD URI\s0 (like
\&\f(CW\*(C`nbd://localhost\*(C'\fR).  The \s-1NBD\s0 server can be local or remote, and
encryption can be used if libnbd was built with encryption support.
Alternately you can use square brackets around a \fBqemu\-nbd\fR\|(8) or
\&\fBnbdkit\fR\|(1) command to run the \s-1NBD\s0 server as a subprocess of nbdcopy.
.PP
For more complex copying operations including converting between disk
formats use \f(CW\*(C`qemu\-img convert\*(C'\fR, see \fBqemu\-img\fR\|(1).
.SH "OPTIONS"
.IX Header "OPTIONS"
.IP "\fB\-\-help\fR" 4
.IX Item "--help"
Display brief command line help and exit.
.IP "\fB\-\-allocated\fR" 4
.IX Item "--allocated"
Normally nbdcopy tries to create sparse output (with holes) if the
destination supports that.  It does this in two ways: either using
extent informtation from the source to copy holes (see
\&\fI\-\-no\-extents\fR), or by detecting runs of zeroes (see \fI\-S\fR).  If you
use \fI\-\-allocated\fR then nbdcopy creates a fully allocated, non-sparse
output on the destination.
.IP "\fB\-C\fR N" 4
.IX Item "-C N"
.PD 0
.IP "\fB\-\-connections=\fRN" 4
.IX Item "--connections=N"
.PD
Set the maximum number of \s-1NBD\s0 connections (\*(L"multi-conn\*(R").  By default
nbdcopy will try to use multi-conn with up to 4 connections if the \s-1NBD\s0
server supports it.  If copying between \s-1NBD\s0 servers then nbdcopy
cannot use multi-conn if either of the servers does not support it.
.IP "\fB\-\-destination\-is\-zero\fR" 4
.IX Item "--destination-is-zero"
.PD 0
.IP "\fB\-\-target\-is\-zero\fR" 4
.IX Item "--target-is-zero"
.PD
Assume the destination is already zeroed.  This allows nbdcopy to skip
copying blocks of zeroes from the source to the destination.  This is
not safe unless the destination device is already zeroed.
(\fI\-\-target\-is\-zero\fR is provided for compatibility with
\&\fBqemu\-img\fR\|(1).)
.IP "\fB\-\-flush\fR" 4
.IX Item "--flush"
Flush writes to ensure that everything is written to persistent
storage before nbdcopy exits.
.IP "\fB\-\-no\-extents\fR" 4
.IX Item "--no-extents"
Normally nbdcopy uses extent metadata to skip over parts of the source
disk which contain holes.  If you use this flag, nbdcopy ignores
extent information and reads everything, which is usually slower.  You
might use this flag in two situations: the source \s-1NBD\s0 server has
incorrect metadata information; or the source has very slow extent
querying so it's faster to simply read all of the data.
.IP "\fB\-p\fR" 4
.IX Item "-p"
.PD 0
.IP "\fB\-\-progress\fR" 4
.IX Item "--progress"
.PD
Display a progress bar.
.IP "\fB\-\-progress=\fR\s-1FD\s0" 4
.IX Item "--progress=FD"
Write a progress bar to the file descriptor \f(CW\*(C`FD\*(C'\fR (a number) in a
format which is easily parsable by other programs.  nbdcopy will
periodically write the string \f(CW"N/100\en"\fR (where N is an integer
between 0 and 100) to the file descriptor.
.Sp
To get nbdcopy to write the progress bar to a file you can use the
following shell commands:
.Sp
.Vb 3
\& exec 3>/tmp/progress
\& nbdcopy \-\-progress=3 ...
\& exec 3>&\-
.Ve
.IP "\fB\-R\fR N" 4
.IX Item "-R N"
.PD 0
.IP "\fB\-\-requests=\fRN" 4
.IX Item "--requests=N"
.PD
Set the maximum number of requests in flight per \s-1NBD\s0 connection.
.IP "\fB\-S\fR N" 4
.IX Item "-S N"
.PD 0
.IP "\fB\-\-sparse=\fRN" 4
.IX Item "--sparse=N"
.PD
Detect all zero blocks of size N (bytes) and make them sparse on the
output.  You can also turn off sparse detection using \fI\-S 0\fR.
The default is 4096 bytes.
.IP "\fB\-\-synchronous\fR" 4
.IX Item "--synchronous"
Force synchronous copying using the \fBlibnbd\fR\|(3) synchronous (\*(L"high
level\*(R") \s-1API.\s0  This is slow but may be necessary for some broken \s-1NBD\s0
servers which cannot handle multiple requests in flight.  This mode is
also used when streaming to and from stdio, pipes and sockets.
.IP "\fB\-T\fR N" 4
.IX Item "-T N"
.PD 0
.IP "\fB\-\-threads=\fRN" 4
.IX Item "--threads=N"
.PD
Use up to N threads for copying.  By default this is set to the number
of processor cores available.
.Sp
Note \fI\-\-threads=0\fR means autodetect and \fI\-\-threads=1\fR means use a
single thread.
.IP "\fB\-V\fR" 4
.IX Item "-V"
.PD 0
.IP "\fB\-\-version\fR" 4
.IX Item "--version"
.PD
Display the package name and version and exit.
.SH "MULTI-CONN, THREADS, REQUESTS IN FLIGHT"
.IX Header "MULTI-CONN, THREADS, REQUESTS IN FLIGHT"
The three options \fI\-\-connections\fR, \fI\-\-threads\fR and \fI\-\-requests\fR are
related and control the amount of parallelism available.  The defaults
should ensure a reasonable amount of parallelism if possible and you
don’t need to adjust them, but this section tries to describe what is
going on.
.PP
Firstly if either side of the copy is streaming to or from stdio, a
pipe, or a socket, or if you use the \fI\-\-synchronous\fR option, then
nbdcopy works in synchronous mode with no parallelism, and nothing
else in this section applies.
.PP
The \fI\-\-connections=N\fR option controls \s-1NBD\s0 multi-conn (see
\&\*(L"Multi-conn\*(R" in \fBlibnbd\fR\|(3)), opening up to N connections to the \s-1NBD\s0
server (or to both \s-1NBD\s0 servers if copying between \s-1NBD\s0 servers).  This
defaults to 4.  The \s-1NBD\s0 servers must support and advertise multi-conn.
For \fBnbdkit\fR\|(1) availability of multi-conn can depend on the plugin.
You can use \fBnbdinfo\fR\|(1) to find out if a particular \s-1NBD\s0 server is
advertising multi-conn.  If the \s-1NBD\s0 server doesn’t advertise
multi-conn then only one connection will be opened regardless of the
\&\fI\-\-connections\fR flag.
.PP
When copying between two \s-1NBD\s0 servers, the number of connections is
limited to the minimum multi-conn supported on both sides.  For the
purposes of this calculation, you can consider local files and block
devices as supporting infinite multi-conn.
.PP
When you run an \s-1NBD\s0 server as a subprocess (using the \f(CW\*(C`[ ... ]\*(C'\fR
syntax) multi-conn cannot be used.
.PP
The \fI\-\-threads=N\fR option allows nbdcopy to start up to N threads
(defaulting to the number of cores).  However nbdcopy cannot use more
threads than the number of \s-1NBD\s0 connections.
.PP
The \fI\-\-requests=N\fR option controls the maximum number of requests in
flight on each \s-1NBD\s0 connection.  This enables the \s-1NBD\s0 server to process
requests in parallel even when multi-conn isn’t available or when
using a single thread.  The default is chosen to allow a reasonable
amount of parallelism without using too much memory.
.PP
Because of this parallelism, nbdcopy does not read or write blocks in
order.  If for some reason you require that blocks are copied in
strict order then you must use \fI\-\-synchronous\fR.
.SH "RUNNING NBD SERVER AS A SUBPROCESS"
.IX Header "RUNNING NBD SERVER AS A SUBPROCESS"
Instead of connecting to an already running server using an \s-1NBD URI,\s0
you can run an \s-1NBD\s0 server as a subprocess using:
.PP
.Vb 1
\& nbdcopy \-\- [ CMD ARGS ... ] ...
.Ve
.PP
This requires the server to support systemd socket activation, which
both \fBqemu\-nbd\fR\|(8) and \fBnbdkit\fR\|(1) support (see also
\&\fBnbd_connect_systemd_socket_activation\fR\|(3)).
.PP
\&\f(CW\*(C`[\*(C'\fR and \f(CW\*(C`]\*(C'\fR must be separate command line parameters.  You will
usually need to use \f(CW\*(C`\-\-\*(C'\fR to stop nbdcopy from misinterpreting \s-1NBD\s0
server flags as nbdcopy flags.  Both the source and destination may be
subprocesses.  nbdcopy cleans up the subprocess on exit.
.PP
Some examples follow.
.SS "nbdcopy \*(-- [ qemu-nbd \-f qcow2 disk.qcow2 ] \- | hexdump \-C"
.IX Subsection "nbdcopy [ qemu-nbd -f qcow2 disk.qcow2 ] - | hexdump -C"
In this example, \fBqemu\-nbd\fR\|(8) is run as a subprocess.  The
subprocess opens \fIdisk.qcow2\fR and exposes it as \s-1NBD\s0 to nbdcopy.
nbdcopy streams this to stdout (\f(CW\*(C`\-\*(C'\fR) into the pipe which is read by
\&\fBhexdump\fR\|(1).
.SS "nbdcopy \*(-- [ qemu-nbd \-f qcow2 disk.qcow2 ] [ nbdkit memory 1G ]"
.IX Subsection "nbdcopy [ qemu-nbd -f qcow2 disk.qcow2 ] [ nbdkit memory 1G ]"
Two subprocesses are created, \fBqemu\-nbd\fR\|(8) as the source and
\&\fBnbdkit\fR\|(1) as the destination.  The qcow2 file is converted to raw
and stored temporarily in the \s-1RAM\s0 disk (\fBnbdkit\-memory\-plugin\fR\|(1)).
.PP
When nbdcopy exits both servers are killed and the \s-1RAM\s0 disk goes away,
so this command has no overall effect, but is useful for testing.
.SH "COMPARING NBD SERVER CONTENT"
.IX Header "COMPARING NBD SERVER CONTENT"
You can use nbdcopy, \fBcmp\fR\|(1) and \fBbash\fR\|(1) process substitution to
compare the content of two \s-1NBD\s0 servers for equality:
.PP
.Vb 1
\& cmp <( nbdcopy nbd://server1 \- ) <( nbdcopy nbd://server2 \- )
.Ve
.PP
Note this tests that the content is logically equal.  It does not
compare the \s-1NBD\s0 metadata such as sparseness (see \fBnbdinfo\fR\|(1)
\&\fI\-\-map\fR option).  Thus for example a run of allocated zeroes in one
server will match a hole in the other.
.SH "SEE ALSO"
.IX Header "SEE ALSO"
\&\fBlibnbd\fR\|(3),
\&\fBnbdfuse\fR\|(1),
\&\fBnbdinfo\fR\|(1),
\&\fBnbdsh\fR\|(1),
\&\fBnbdkit\fR\|(1),
\&\fBqemu\-img\fR\|(1).
.SH "AUTHORS"
.IX Header "AUTHORS"
Richard W.M. Jones
.SH "COPYRIGHT"
.IX Header "COPYRIGHT"
Copyright (C) 2020 Red Hat Inc.
.SH "LICENSE"
.IX Header "LICENSE"
This library is free software; you can redistribute it and/or
modify it under the terms of the \s-1GNU\s0 Lesser General Public
License as published by the Free Software Foundation; either
version 2 of the License, or (at your option) any later version.
.PP
This library is distributed in the hope that it will be useful,
but \s-1WITHOUT ANY WARRANTY\s0; without even the implied warranty of
\&\s-1MERCHANTABILITY\s0 or \s-1FITNESS FOR A PARTICULAR PURPOSE.\s0  See the \s-1GNU\s0
Lesser General Public License for more details.
.PP
You should have received a copy of the \s-1GNU\s0 Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, \s-1MA 02110\-1301 USA\s0