When desired, the build daemon can offload derivation builds to
other machines running Guix, using the
hook4. When that
feature is enabled, a list of user-specified build machines is read from
/etc/guix/machines.scm; every time a build is requested, for
guix build, the daemon attempts to offload it to one
of the machines that satisfy the constraints of the derivation, in
particular its system type—e.g., x86_64-linux. Missing
prerequisites for the build are copied over SSH to the target machine,
which then proceeds with the build; upon success the output(s) of the
build are copied back to the initial machine.
The /etc/guix/machines.scm file typically looks like this:
(list (build-machine (name "eightysix.example.org") (system "x86_64-linux") (host-key "ssh-ed25519 AAAAC3Nza…") (user "bob") (speed 2.)) ;incredibly fast! (build-machine (name "meeps.example.org") (system "mips64el-linux") (host-key "ssh-rsa AAAAB3Nza…") (user "alice") (private-key (string-append (getenv "HOME") "/.ssh/identity-for-guix"))))
In the example above we specify a list of two build machines, one for
x86_64 architecture and one for the
In fact, this file is—not surprisingly!—a Scheme file that is
evaluated when the
offload hook is started. Its return value
must be a list of
build-machine objects. While this example
shows a fixed list of build machines, one could imagine, say, using
DNS-SD to return a list of potential build machines discovered in the
local network (see Guile-Avahi in Using
Avahi in Guile Scheme Programs). The
build-machine data type is
This data type represents build machines to which the daemon may offload builds. The important fields are:
The host name of the remote machine.
The system type of the remote machine—e.g.,
The user account to use when connecting to the remote machine over SSH. Note that the SSH key pair must not be passphrase-protected, to allow non-interactive logins.
This must be the machine’s SSH public host key in OpenSSH format. This is used to authenticate the machine when we connect to it. It is a long string that looks like this:
ssh-ed25519 AAAAC3NzaC…mde+UhL firstname.lastname@example.org
If the machine is running the OpenSSH daemon,
sshd, the host
key can be found in a file such as
If the machine is running the SSH daemon of GNU lsh,
lshd, the host key is in /etc/lsh/host-key.pub or a
similar file. It can be converted to the OpenSSH format using
lsh-export-key (see Converting keys in LSH Manual):
$ lsh-export-key --openssh < /etc/lsh/host-key.pub ssh-rsa AAAAB3NzaC1yc2EAAAAEOp8FoQAAAQEAs1eB46LV…
A number of optional fields may be specified:
Port number of SSH server on the machine.
The SSH private key file to use when connecting to the machine, in OpenSSH format. This key must not be protected with a passphrase.
Note that the default value is the private key of the root account. Make sure it exists if you use the default.
The SSH-level compression methods and compression level requested.
Note that offloading relies on SSH compression to reduce bandwidth usage when transferring files to and from build machines.
File name of the Unix-domain socket
guix-daemon is listening
to on that machine.
The number of builds that may run in parallel on the machine.
A “relative speed factor”. The offload scheduler will tend to prefer machines with a higher speed factor.
A list of strings denoting specific features supported by the machine.
An example is
"kvm" for machines that have the KVM Linux modules
and corresponding hardware support. Derivations can request features by
name, and they will be scheduled on matching build machines.
guile command must be in the search path on the build
machines. In addition, the Guix modules must be in
$GUILE_LOAD_PATH on the build machine—you can check whether
this is the case by running:
ssh build-machine guile -c "'(use-modules (guix config))'"
There is one last thing to do once machines.scm is in place. As explained above, when offloading, files are transferred back and forth between the machine stores. For this to work, you first need to generate a key pair on each machine to allow the daemon to export signed archives of files from the store (see Invoking guix archive):
# guix archive --generate-key
Each build machine must authorize the key of the master machine so that it accepts store items it receives from the master:
# guix archive --authorize < master-public-key.txt
Likewise, the master machine must authorize the key of each build machine.
All the fuss with keys is here to express pairwise mutual trust relations between the master and the build machines. Concretely, when the master receives files from a build machine (and vice versa), its build daemon can make sure they are genuine, have not been tampered with, and that they are signed by an authorized key.
To test whether your setup is operational, run this command on the master node:
# guix offload test
This will attempt to connect to each of the build machines specified in /etc/guix/machines.scm, make sure Guile and the Guix modules are available on each machine, attempt to export to the machine and import from it, and report any error in the process.
If you want to test a different machine file, just specify it on the command line:
# guix offload test machines-qualif.scm
Last, you can test the subset of the machines whose name matches a regular expression like this:
# guix offload test machines.scm '\.gnu\.org$'
To display the current load of all build hosts, run this command on the main node:
# guix offload status