Bootstrapping in our context refers to how the distribution gets built “from nothing”. Remember that the build environment of a derivation contains nothing but its declared inputs (see Introduction). So there’s an obvious chicken-and-egg problem: how does the first package get built? How does the first compiler get compiled? Note that this is a question of interest only to the curious hacker, not to the regular user, so you can shamelessly skip this section if you consider yourself a “regular user”.
The GNU system is primarily made of C code, with libc at its core. The
GNU build system itself assumes the availability of a Bourne shell and
command-line tools provided by GNU Coreutils, Awk, Findutils, ‘sed’, and
‘grep’. Furthermore, build programs—programs that run
make, etc.—are written in Guile Scheme
(see Derivations). Consequently, to be able to build anything at
all, from scratch, Guix relies on pre-built binaries of Guile, GCC,
Binutils, libc, and the other packages mentioned above—the
These bootstrap binaries are “taken for granted”, though we can also re-create them if needed (more on that later).
The figure above shows the very beginning of the dependency graph of the
distribution, corresponding to the package definitions of the
packages bootstrap) module. At this level of detail, things are
slightly complex. First, Guile itself consists of an ELF executable,
along with many source and compiled Scheme files that are dynamically
loaded when it runs. This gets stored in the guile-2.0.7.tar.xz
tarball shown in this graph. This tarball is part of Guix’s “source”
distribution, and gets inserted into the store with
(see The Store).
But how do we write a derivation that unpacks this tarball and adds it
to the store? To solve this problem, the
derivation—the first one that gets built—uses
bash as its
builder, which runs
build-bootstrap-guile.sh, which in turn calls
tar to unpack the tarball. Thus, bash, tar,
xz, and mkdir are statically-linked binaries, also part of
the Guix source distribution, whose sole purpose is to allow the Guile
tarball to be unpacked.
guile-bootstrap-2.0.drv is built, we have a functioning
Guile that can be used to run subsequent build programs. Its first task
is to download tarballs containing the other pre-built binaries—this
is what the
.tar.xz.drv derivations do. Guix modules such as
ftp-client.scm are used for this purpose. The
module-import.drv derivations import those modules in a directory
in the store, using the original layout. The
module-import-compiled.drv derivations compile those modules, and
write them in an output directory with the right layout. This
corresponds to the
#:modules argument of
build-expression->derivation (see Derivations).
Finally, the various tarballs are unpacked by the
etc., at which point we have a working C tool chain.
Bootstrapping is complete when we have a full tool chain that does not
depend on the pre-built bootstrap tools discussed above. This
no-dependency requirement is verified by checking whether the files of
the final tool chain contain references to the /gnu/store
directories of the bootstrap inputs. The process that leads to this
“final” tool chain is described by the package definitions found in
(gnu packages base) module.
The first tool that gets built with the bootstrap binaries is GNU Make, which is a prerequisite for all the following packages. From there Findutils and Diffutils get built.
Then come the first-stage Binutils and GCC, built as pseudo cross
--target equal to
--host. They are
used to build libc. Thanks to this cross-build trick, this libc is
guaranteed not to hold any reference to the initial tool chain.
From there the final Binutils and GCC are built. GCC uses
from the final Binutils, and links programs against the just-built libc.
This tool chain is used to build the other packages used by Guix and by
the GNU Build System: Guile, Bash, Coreutils, etc.
And voilà! At this point we have the complete set of build tools that
the GNU Build System expects. These are in the
variables of the
(gnu packages base) module, and are implicitly
used by any package that uses
gnu-build-system (see Defining Packages).
Because the final tool chain does not depend on the bootstrap binaries,
those rarely need to be updated. Nevertheless, it is useful to have an
automated way to produce them, should an update occur, and this is what
(gnu packages make-bootstrap) module provides.
The following command builds the tarballs containing the bootstrap binaries (Guile, Binutils, GCC, libc, and a tarball containing a mixture of Coreutils and other basic command-line tools):
guix build bootstrap-tarballs
The generated tarballs are those that should be referred to in the
(gnu packages bootstrap) module mentioned at the beginning of
Still here? Then perhaps by now you’ve started to wonder: when do we reach a fixed point? That is an interesting question! The answer is unknown, but if you would like to investigate further (and have significant computational and storage resources to do so), then let us know.