.TH MIRRORRIB 1 "2021-09-04" "0.14.4.2" "Debian, the Universal Operating System"
.SH NAME
mirrorrib \- locally mirror a Debian release, including backports
.SH SYNOPSIS
.SY mirrorrib
.RI [ OPTIONS ]
.I DEBIAN-VER
.IR ARGUMENTS \&.\|.\|.\&
.br
.SY mirrorrib
.\".OP \-lsiIBt?\-
.OP \-ls?\-
.OP \-p FILE
.RB { 10. \fIN\fR| 11. \fIN\fR| 12. \fIN\fR}
.I CD-REV ARCH OLD TARGET ISO MIRROR
.RI [ LANGS \&.\|.\|.\&]
.YS
.SH DESCRIPTION
.\" Except for the paragraph on Debian-CD, the description has been copied
.\" verbatim, with manual reformatting, from debian/control.
.P
Debian releases a major revision of its operating system about every two
years and a minor revision approximately quarterly, but these revisions
exclude Debian backports.
Debian releases backports only on a rolling basis like sid.
.P
.B mirrorrib
is for Debian users who want an approximately quarterly, stable revision of
backports to accompany the approximately quarterly, stable revision of the
rest of the operating system\(emwith both revisions dated as of the same
date.
.P
.B mirrorrib
reproducibly assembles a stable backports revision and release to accompany
a stable regular revision and release.
It downloads the matched pair of releases with all their packages and
associated files, mirroring the pair together to your hard drive.
After running
.B mirrorrib
and configuring
.I /etc/apt/sources.list
to access the local repository
.B mirrorrib
has assembled, one no longer needs a live network connection to update or
reinstall one's system to Debian stable\(emnot even if the update or
reinstallation requires access to backports.
.P
In addition to the matched pair of releases,
.B mirrorrib
also mirrors the standard, bootable system-installation image,
.IR netinst.iso ,
from the Debian-CD archive.
.P
.BR mirrorrib 's
name stands for \(lqMIRROR Release Including Backports.\(rq
.SH ARGUMENTS
.P
The first argument to
.BR mirrorrib ,
the
.IR DEBIAN-VER ,
specifies the stable version of Debian whose release
.B mirrorrib
is to fetch.
The arrangement and number of the other arguments depend on the value of
the first argument.
(At present, only Debian
.BI 10. N\fR,
.BI 11. N
and
.BI 12. N
are supported.
.B mirrorrib
arranges its command line similarly whether for Debian
.BI 10. N\fR,
for Debian
.BI 11. N
or for Debian
.BI 12. N\fR.
When future support for Debian
.BI 13. N
arrives, though,
.B mirrorrib
might arrange its command line for that Debian version differently.
Check this page once future support arrives.)
.TP
.I DEBIAN-VER
The number of the stable version of Debian whose release
.B mirrorrib
is to fetch (for example
.BR 11.1
to fetch the .1 release of Debian 11 bullseye).
.TP
.I CD-REV
The Debian-CD revision number (usually
.BR 0 ,
for example for Debian-CD revision 11.1.0).
.TP
.I ARCH
The computer architecture
.RB ( amd64 ,
for example) for which the release is to be fetched.
.TP
.I OLD
To avoid unnecessary downloading, an old Debian repository already on your
hard drive
.RB ( /home/jsmith/debian-11.0 ,
for example), from which
.B mirrorrib
can try to copy or hard-link files already downloaded.
Enter
.B /dev/null
if no such repository is available, but beware:
.B /dev/null
results in a huge download, possibly of hundreds of GiB, which is
presumably not what you wanted if you had run
.B mirrorrib
before and, thus, already possessed most of the needed files!  (Note that
.B /dev/null
does not cause files to be downloaded
.I to
.BR /dev/null ,
but to be copied locally
.I from
.BR /dev/null ,
which is to say, not copied locally at all.)
.TP
.I TARGET
The location
.RB ( /home/jsmith/debian-11.1 ,
for example),
which must be a directory that does not yet exist, at which
.B mirrorrib
is to install the new, matched pair of Debian releases\(emwith all their
packages and associated files\(emas a proper Debian repository.
You may find it useful to put this on a different hard-disk partition (as
.BR /other-partition/debian-11.1 ,
for example) than the one onto which the operating system is installed, for
doing so lets the new repository be used during a future
wipe-and-reinstallation of the machine.
.TP
.I ISO
The location
.RB ( /home/jsmith/debian-cd-11.1.0
or
.BR /other-partition/debian-cd-11.1.0 ,
for example), which must be a directory that does not yet exist, at which
.B mirrorrib
is to install the new Debian-CD repository.
(Note that only your architecture's
.I netinst.iso
along with its associated checksums are fetched to the new Debian-CD
repository, for the Debian Project's debian-cd service specifies no
particular release suite and
.I netinst.iso
is the minimal, standard ISO image one needs to boot the installation of a
fresh Debian system\(emduring which fresh installation one can rely on the
new, local repository to supply packages and associated files instead of
relying on a network connection.
Alternate ISO images and various other nonessential Debian-CD files exist,
but because a canonical, reproducible selection among those is impossible,
none of those other images or files is fetched.)
.TP
.I MIRROR
The regular Debian mirror from which
.B mirrorrib
is to download
.RB ( mirror.example.org ,
for example).
Choose from the list at
.I https://www.debian.org/mirror/
a single mirror that serves both
.I debian/
and
.IR debian-cd/
for your architecture via the
.I rsync
protocol.
(If unsure about
.IR rsync ,
then just pick a mirror and try.
Most Debian mirrors support
.IR rsync
and if the one you have chosen does not, then
.B mirrorrib
will detect that and abort before starting the big download.
If you have issued the
.B \-s
option, then the mirror does not matter, so you can just enter
.B localhost
in that case.)
.TP
.IR LANGS \&.\|.\|.\&
A space-separated list of the languages whose package-description
Translation files are to be fetched (for example,
.B en fr de de_DE
for English, French, German and the German of Germany; normally, you should
at least list
.B en
here).
Enter
.B @
to fetch all available languages.
.P
The examples use absolute pathnames but of course relative pathnames
.RB ( debian-11.1
or
.BR foo/debian-11.1 ,
for example) work just as well.
.SH OPTIONS
.TP
.B \-l
Save local disk space by hard-linking pool files from
.I OLD
to
.I TARGET
where practicable.
(Recommended.
Note however that a hard-linked file is
.\" The next line's two commas would visually clash if one were .I and the
.\" other, .R.
.I one, single file,
accessible in two or more different locations.
If for some reason
.B mirrorrib
found it necessary to update a hard-linked file\(emmost likely by
correcting the file's timestamp\(emthen it would update the file equally in
.I OLD
as well as in
.IR TARGET .)
.TP
.B \-s
Change nothing, but print checksums as listed in the release's control
files (such as
.IR Packages ,
.I Sources
and
.I SHA256SUMS
files).
This option does not care about the
.IR OLD ,
.I ISO
and
.I MIRROR
parameters, so you can enter
.B /dev/null
for each of the first two and
.B localhost
for the last if you wish.
However, it does require a correct
.I LANGS
list.
.\".\" Experimental:
.\".TP
.\".B \-i
.\"Fetch only ISO CD boot images.
.\".TP
.\".B \-I
.\"Skip ISO CD boot images.
.\".TP
.\".B \-B
.\"Omit backports.
.TP
.B \-?
Print a help message.
Do nothing else.
.TP
.B \-\-
Disable further options processing.
.TP
.B \-p\  \fIFILE
Set
.IR FILE 's
modification timestamp to indicate the date and time of the snapshot upon
which
.B mirrorrib
has relied.
(The
.I FILE
is not a file
.B mirrorrib
downloads but is any file you name, usually a nonexistent or empty file.
Naming such a file is useful if you wish to identify at
.I https://snapshot.debian.org/archive/debian/
the specific snapshot
.B mirrorrib
has referenced.
If the file you name exists, then
.B mirrorrib
timestamps it but does not otherwise alter it.
If the file does not exist, then
.B mirrorrib
creates and timestamps it.)
.P
Besides these,
.B mirrorrib
also accepts
.B \-\-help
or
.BR \-\-version ,
provided that the option immediately follows the command.
These do what you expect.
The
.B \-\-help
option does the same as the
.B \-?
option does.
.SH EXIT STATUS
.TP
.B 0
Successful program execution.
.TP
Any other integer
Failure.
.RB [ mirrorrib
itself always generates the exit status
.B 1
upon failure, but the various other programs mirrorrib calls during its
execution, such as
.BR rsync (1),
can issue other integers.
In most cases, an error message will be printed to
.I stderr
to indicate the failure's cause.]
.SH ENVIRONMENT
.P
Summary:  except for
.BR LC_MESSAGES ,
neither the environment nor the
.B umask
matters much to
.BR mirrorrib .
Details follow.
.TP
.B LC_MESSAGES
The locale, as
.B fr_FR.UTF-8
(for the French language of France)
or
.B de_AT.UTF-8
(for the German language of Austria) for example, to which
.BR mirrorrib 's
diagnostic messages are translated.
.TP
.B LC_ALL LANG
Alternatives to
.B LC_MESSAGES
that have exactly the same effect on
.B mirrorrib LC_MESSAGES
has.
As
.BR locale (7)
explains,
.B LC_ALL
supersedes
.B LC_MESSAGES
unless
.B LC_ALL
is null, whereas
.B LANG
serves as a fallback in case both of the others are null.
If you need localization, then your
.B LANG
is probably already set to the right default, so you won't need to worry
about the other variables; but if your
.B LANG
is null or wrong and you are unsure of what to do, then just set
.BR LC_MESSAGES
and leave the other two alone.
The most straightforward way to do this is to set the variable by prefixing
the command line.
For example,
.B LC_MESSAGES=fr_FR.UTF-8
.BR mirrorrib \&.\|.\|.\|.\&
(Note that the last works only if you have generated the
.B fr_FR.UTF-8
locale on your system.
If you haven't and you wish to, then issue
.B dpkg-reconfigure locales
as root.)
.TP
.B TZ
No effect.
.B mirrorrib
uses UTC, regardless of the time zone to which
.B TZ
is set.
(The matter of times zones is mentioned because the matter confuses some
inexperienced Debian users and because timestamps are significant to
.BR mirrorrib .
Internally, the Debian operating system keeps time solely in UTC,
regardless of the time zone to which
.B TZ
is set.
Files are stamped in UTC, not only by
.B mirrorrib
but by other programs, as well.
The
.B TZ
affects how time is
.I displayed
by certain tools like
.BR date (1),
but not how time is stored.
The point is that, if you transferred a repository assembled by
.B mirrorrib
in one time zone to a second machine in another time zone, then the second
machine could not\(emby examining the transferred repository's
timestamps\(eminfer the time zone in which the repository had been
assembled.
The second machine would find only timestamps in correct UTC.
In other words, results are reliable.
You need not worry about it.)
.TP
Other variables
No effect.  Internally,
.B mirrorrib
launches several other programs but, except for the aforementioned
.BR LC_MESSAGES ,
.B mirrorrib
does not pass the user's environment variables to them.
.P
Though the user's
.B umask
setting is not part of the environment as such
(for a new Linux process inherits its
.BR umask ,
along with its working directory, directly without regard to the
environment via the kernel's process-launching mechanism), the
.B umask
is mentioned here because
.B mirrorrib
respects it\(emmomentarily.
After downloading,
.B mirrorrib
resets the modes of downloaded files and symlinks to 0644 and of downloaded
directories to 0755, anyway, so in the end the
.B umask
hardly matters in normal use cases.  As far as the maintainer is aware, the
.B umask
affects no
.I persistent
production of
.B mirrorrib
except the
.B \-p
option's timestamp.
Nevertheless, the
.BR umask 's
usual default value on a Debian system is 0022, so if you wish to set a
specific value before launching
.B mirrorrib
(or almost any other program), then you can issue
.B umask 0022
first.
.P
But it probably isn't necessary.
.SH FILES
.TP
.I /etc/mirrorrib.conf
Systemwide configuration of
.BR mirrorrib .
(You probably won't need to touch it.)
.TP
.I /etc/apt/sources.list
List of locations from which
.BR apt-get (8)
is to fetch packages.
.SH CONFORMING TO
.IP \(bu
Debian 10 buster
.IP \(bu
Debian 11 bullseye
.IP \(bu
Debian 12 bookworm (tentative)
.\" .IP \(bu
.\" Debian 13 trixie (planned)
.SH USAGE WITH APT
.P
Debian's rolling backports releases are each marked to expire a week after
issue, whereas a
.B mirrorrib
repository expects to remain in service about three months, until the next
stable release arrives.
APT will object if asked to update according to an expired backports
.I Release
file.
After
.B mirrorrib
has finished its run and you have suitably configured
.IR /etc/apt/sources.list ,
you can overcome APT's objection by issuing the command
.IP
.EX
.B apt-get \-o Acquire::Check-Valid-Until=false update
.EE
.P
as root.
.P
If backports use with APT is generally unfamiliar to you, then refer to the
.B \-t
option on the manual page of
.BR apt-get (8).
.P
As far as
.I /etc/apt/sources.list
goes, if you are already as familiar with that file as many Debian users
are, then you can configure the file however you like.
However, if you are unsure, then, after backing up the existing file, you
might try letting the file consist of several lines resembling these:
.P
.RS 4n
.EX
.SM deb\ \ \ \ \ file:///home/jsmith/debian-11.1/ bullseye main contrib non-free
.br
.SM deb-src\ file:///home/jsmith/debian-11.1/ bullseye main contrib non-free
.br
.SM deb\ \ \ \ \ file:///home/jsmith/debian-11.1/ bullseye-backports main contrib non-free
.br
.SM deb-src\ file:///home/jsmith/debian-11.1/ bullseye-backports main contrib non-free
.EE
.RE
.P
Notice that the repository
.B mirrorrib
has installed makes it unnecessary to list an online mirror, though if you
require rolling security updates then, of course, you still must list the
Debian Project's security mirror for that.
(In the latter case, consult
.I https://www.debian.org/security/
for further instructions.)
.SH RELIANCE ON THE DEBIAN PROJECT'S SNAPSHOT SERVICE
.P
.B
mirrorrib
relies on the Debian Project's snapshot service at
.I https://snapshot.debian.org/
for several purposes, mainly to serve backports'
.I dists/
and, in
.IR pool/ ,
to serve specific backports packages that have expired from the regular
mirrors.
.P
.I The snapshot service is not a mirror.
Therefore, it is important that the user not abuse it.
Use is logged.
Abusers can be banned, but if like most Debian users your habit is to be a
good netizen, then the prospect of bans has little to do with you.
Therefore, to moderate the load (and indeed to unclog your own Internet
connection), please consider specifying an
.I OLD
repository when invoking
.B mirrorrib
if feasible.
.P
(Kindly do not contact the maintainer regarding bans.
The maintainer does not issue bans, cannot revoke them, and does not know
you well enough to advocate on your behalf.
If the maintainer understands, there was one instance in which the
persistent actions of a single abuser forced the snapshot service to ban
the entirety of a major Internet infrastructure company whose name would be
familiar to you if the name were mentioned here, that the snapshot service
could remain functional for the rest of us.
So don't do that.)
.P
For information, rather than serving files via
.I rsync
as the regular mirror does, snapshot serves files only via
.IR http(s) ,
which leaves
.B mirrorrib
locally to treat several complications regarding symlinks, directories and
timestamps.
.B mirrorrib
is programmed to treat these complications automatically, so the details
need not concern the user; except that, to achieve reproducibility,
.B mirrorrib
is forced locally to stamp a standardized date and time on certain symlinks
and directories (mainly regarding backports) rather than duplicating all
timestamps off a regular mirror as one otherwise would do.
.SH THE REPOSITORY'S SIZE
.P
Even for a single stable release for a single architecture, the local
repository
.B mirrorrib
assembles is big.
The manual page you are reading cannot tell you in advance how big, but, at
this writing, the last time the author ran
.BR mirrorrib ,
himself, for Debian bullseye
.\" 11.1
11.0
amd64, the repository proved to be 165 GiB in size.
The size of backports' contribution fluctuates but, by the time you read
this manual page, the combined repository including backports will probably
have grown even larger than the 165 GiB.
.P
Fortunately, during your second and subsequent invocations of
.BR mirrorrib ,
or even during your first invocation if you already happen to have a local
repository on your hard drive by another tool, the
.B \-l
option helps to conserve drive space.
.SH THE @ LANGUAGE
.P
When the author runs
.BR mirrorrib ,
himself,
he lists on the command line only
.B en
for English plus the standard abbreviations for one or two other languages
he is able to read.
However, this is a matter of preference.
Compared to the size of the overall repository, the size of Debian's
.I Translation
files is not very significant, so if you prefer to order
.B @
for all available languages, go ahead and do it.
.SH WIRELESS FIRMWARE TROUBLES
.P
Refer to
.IR /usr/share/doc/mirrorrib/NON-FREE-FIRMWARE .
.SH REPRODUCIBLE RESULTS
.P
.B mirrorrib
is designed to deliver strictly reproducible results, even as regards
file-, symlink- and directory-modification timestamps.
(If you should notice a difference between results obtained on one date or
machine and results obtained on another date or machine, if you are running
the latest
.BR mirrorrib ,
and if you can show how to reproduce the behavior, then kindly report the
bug via Debian's Bug Tracking System with normal priority.)
.SH TENTATIVE SUPPORT FOR DEBIAN 12
.P
When the present version of
.B mirrorrib
was prepared, Debian 12 was still in testing.
At that time, the present version of
.B mirrorrib
seemed ready to handle Debian 12 correctly.
Nevertheless,
.BR mirrorrib 's
maintainer cannot guarantee that, when Debian 12 stable arrives, the
present version of
.B mirrorrib
will still handle it correctly.
Indeed, depending on the finalized details of Debian 12 (and on
the soundness of
.BR mirrorrib 's
maintainer's forward-looking grasp of those details), there is a
significant probability that
.B mirrorrib
will not handle it correctly.
If it does not seem to, then look for an updated
.I mirrorrib
package in Debian's
.I backports
archive online.
.SH PATIENCE ON THE DAY OF A NEW RELEASE
.P
.B mirrorrib
will probably not do what you want during the first 48 hours or so
following the Debian Project's approximately quarterly announcement of a
new stable release.
The reason is that time is needed for Debian's Images Team to build and
upload ISO CD installation images, for hundreds or thousands of pool and
dists files to propagate to the mirrors, and for sufficient numbers of pre-
and post-release snapshots to appear on
.I snapshot.debian.org
to afford
.B mirrorrib
enough information to deduce with confidence which is the optimal snapshot
for
.B mirrorrib
to reference.
If
.B mirrorrib
fails and the release is new, then try again tomorrow.
Forty-eight hours probably suffice.
The author has never noticed the overall delay to take longer than a week.
.\" .SH LIMITATIONS
.SH EXAMPLES
.P
If an old repository is locally present and English, French, German and the
German of Germany are wanted,
.P
.RS
.EX
.B mirrorrib \-lp /home/jsmith/snapshot.stamp 11.1 0 amd64 \e
.br
.RS 1n
.B /home/jsmith/debian-11.0 \e
.br
.B /home/jsmith/debian-11.1 \e
.br
.B /home/jsmith/debian-cd-11.1.0 \e
.br
.B mirror.example.org en fr de de_DE
.RE
.EE
.RE
.P
Alternately, if no old repository is locally present and all available
languages are wanted (caution:
because no old repository, a huge download will ensue!),
.P
.RS
.EX
.B mirrorrib \-p /home/jsmith/snapshot.stamp 11.1 0 amd64 \e
.br
.RS 1n
.B /dev/null \e
.br
.B /home/jsmith/debian-11.1 \e
.br
.B /home/jsmith/debian-cd-11.1.0 \e
.br
.B mirror.example.org @
.RE
.EE
.RE
.P
Subsequently, user
.IR jsmith ,
who has run the program, can examine the snapshot timestamp by
.IP
.EX
.B TZ=UTC stat \-c%y /home/jsmith/snapshot.stamp
.EE
.P
If user
.I jsmith
has sufficient hard-drive space available, then he or she can pack up
pristine copies of the new repositories by
.IP
.EX
.B cd /home/jsmith
.br
.B tar \-\-owner=0 \-\-group=0 \-cf debian-11.1{.tar,}
.br
.B tar \-\-owner=0 \-\-group=0 \-cf debian-cd-11.1.0{.tar,}
.EE
.P
(Note that it is probably unnecessary and undesirable for user
.I jsmith
to compress the
.I .tar
files
as
.I .tar.xz
or the like, for most of the content is already compressed.
Note also that user
.IR jsmith 's
language selection influences only which
.I Translation
files are retrieved.
All packages belonging to the release for the selected architecture,
including culture-specific packages, are retrieved in any case.)
.P
Before running
.BR mirrorrib ,
if user
.I jsmith
has old files in two different old repositories, then he or she can issue
.P
.RS
.EX
.B for A in $(find old2 \-type f \-printf \(aq%P\en\(aq); do
.br
.RS 4n
.B [ \-e \(dqold1/$A\(dq ] || {
.br
.RS 4n
.B mkdir \-pv \(dqold1/$(dirname \(dq$A\(dq)\(dq \e
.br
.RS 1n
.B && ln \-v \(dq$(realpath \-e old2)/$A\(dq \e
.br
.B \(dq$(realpath \-e old1)/$A\(dq
.RE
.RE
.B }
.RE
.B done
.EE
.RE
.P
to merge the files of
.B old2/
into
.BR old1/ .
.P
After running
.BR mirrorrib ,
if user
.I jsmith
wishes to verify, against the
.I Release
files
.B mirrorrib
has fetched, the rest of the files it has fetched, then
user
.I jsmith
can issue
.P
.RS
.EX
.B cd /home/jsmith/debian-11.1
.br
.B sha256sum \-c \-\-quiet <(mirrorrib \-s 11.1 0 amd64 \e
.RS 1n
.B /dev/null . /dev/null localhost @)
.br
.RE
.B cd /home/jsmith/debian-cd-11.1.0/11.1.0/amd64/iso-cd
.br
.B sha256sum \-c \-\-ignore-missing SHA256SUMS
.EE
.RE
.P
Of the pair of
.BR sha256sum (1)
commands, the first checks the main repository, takes perhaps half an hour,
and is quiet if it finds no problems.
The second checks the ISO CD repository's
.I netinst.iso
image, is much quicker, and prints a single line to verify the image.
.\"The
.\".BR find (1)
.\"command looks for detritus inadvertently left behind during the
.\"download, which is improbable but might occur if the download was
.\"interrupted by power failure.
Observe that
.BR mirrorrib 's
.B \-s
option requires a correct
.I LANGS
list, so the example's
.B @
is correct only if
.B @
was also issued earlier, during
.BR mirrorrib 's
main run.
.SH AUTHOR
.P
.MT thb@debian.org
Thaddeus H. Black
.ME
.SH SEE ALSO
.P
.BR aptly (1),
.BR chmod (1),
.BR date (1),
.BR debmirror (1),
.BR dirname (1),
.BR df (1),
.BR du (1),
.BR find (1),
.BR ln (1),
.BR mkdir (1),
.BR realpath (1),
.BR rsync (1),
.BR sha256sum (1),
.BR stat (1),
.BR tar (1),
.BR wget (1),
.BR null (4),
.BR apt-conf (5),
.BR sources.list (5),
.BR locale (7),
.BR apt-get (8),
.BR dpkg-reconfigure (8)