Previous: , Up: Setting Up the Daemon   [Contents][Index]


2.4.2 Using the Offload Facility

When desired, the build daemon can offload derivation builds to other machines running Guix, using the offload build hook4. When that feature is enabled, a list of user-specified build machines is read from /etc/guix/machines.scm; every time a build is requested, for instance via guix build, the daemon attempts to offload it to one of the machines that satisfy the constraints of the derivation, in particular its system type—e.g., x86_64-linux. Missing prerequisites for the build are copied over SSH to the target machine, which then proceeds with the build; upon success the output(s) of the build are copied back to the initial machine.

The /etc/guix/machines.scm file typically looks like this:

(list (build-machine
        (name "eightysix.example.org")
        (system "x86_64-linux")
        (host-key "ssh-ed25519 AAAAC3Nza…")
        (user "bob")
        (speed 2.))     ;incredibly fast!

      (build-machine
        (name "meeps.example.org")
        (system "mips64el-linux")
        (host-key "ssh-rsa AAAAB3Nza…")
        (user "alice")
        (private-key
         (string-append (getenv "HOME")
                        "/.ssh/identity-for-guix"))))

In the example above we specify a list of two build machines, one for the x86_64 architecture and one for the mips64el architecture.

In fact, this file is—not surprisingly!—a Scheme file that is evaluated when the offload hook is started. Its return value must be a list of build-machine objects. While this example shows a fixed list of build machines, one could imagine, say, using DNS-SD to return a list of potential build machines discovered in the local network (see Guile-Avahi in Using Avahi in Guile Scheme Programs). The build-machine data type is detailed below.

Data Type: build-machine

This data type represents build machines to which the daemon may offload builds. The important fields are:

name

The host name of the remote machine.

system

The system type of the remote machine—e.g., "x86_64-linux".

user

The user account to use when connecting to the remote machine over SSH. Note that the SSH key pair must not be passphrase-protected, to allow non-interactive logins.

host-key

This must be the machine’s SSH public host key in OpenSSH format. This is used to authenticate the machine when we connect to it. It is a long string that looks like this:

ssh-ed25519 AAAAC3NzaC…mde+UhL hint@example.org

If the machine is running the OpenSSH daemon, sshd, the host key can be found in a file such as /etc/ssh/ssh_host_ed25519_key.pub.

If the machine is running the SSH daemon of GNU lsh, lshd, the host key is in /etc/lsh/host-key.pub or a similar file. It can be converted to the OpenSSH format using lsh-export-key (see Converting keys in LSH Manual):

$ lsh-export-key --openssh < /etc/lsh/host-key.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAAEOp8FoQAAAQEAs1eB46LV…

A number of optional fields may be specified:

port (default: 22)

Port number of SSH server on the machine.

private-key (default: ~/.ssh/id_rsa)

The SSH private key file to use when connecting to the machine, in OpenSSH format.

compression (default: "zlib@openssh.com,zlib")
compression-level (default: 3)

The SSH-level compression methods and compression level requested.

Note that offloading relies on SSH compression to reduce bandwidth usage when transferring files to and from build machines.

daemon-socket (default: "/var/guix/daemon-socket/socket")

File name of the Unix-domain socket guix-daemon is listening to on that machine.

parallel-builds (default: 1)

The number of builds that may run in parallel on the machine.

speed (default: 1.0)

A “relative speed factor”. The offload scheduler will tend to prefer machines with a higher speed factor.

features (default: '())

A list of strings denoting specific features supported by the machine. An example is "kvm" for machines that have the KVM Linux modules and corresponding hardware support. Derivations can request features by name, and they will be scheduled on matching build machines.

The guile command must be in the search path on the build machines. In addition, the Guix modules must be in $GUILE_LOAD_PATH on the build machine—you can check whether this is the case by running:

ssh build-machine guile -c "'(use-modules (guix config))'"

There is one last thing to do once machines.scm is in place. As explained above, when offloading, files are transferred back and forth between the machine stores. For this to work, you first need to generate a key pair on each machine to allow the daemon to export signed archives of files from the store (see Invoking guix archive):

# guix archive --generate-key

Each build machine must authorize the key of the master machine so that it accepts store items it receives from the master:

# guix archive --authorize < master-public-key.txt

Likewise, the master machine must authorize the key of each build machine.

All the fuss with keys is here to express pairwise mutual trust relations between the master and the build machines. Concretely, when the master receives files from a build machine (and vice versa), its build daemon can make sure they are genuine, have not been tampered with, and that they are signed by an authorized key.

To test whether your setup is operational, run this command on the master node:

# guix offload test

This will attempt to connect to each of the build machines specified in /etc/guix/machines.scm, make sure Guile and the Guix modules are available on each machine, attempt to export to the machine and import from it, and report any error in the process.

If you want to test a different machine file, just specify it on the command line:

# guix offload test machines-qualif.scm

Last, you can test the subset of the machines whose name matches a regular expression like this:

# guix offload test machines.scm '\.gnu\.org$'

Footnotes

(4)

This feature is available only when Guile-SSH is present.


Previous: , Up: Setting Up the Daemon   [Contents][Index]