6.19.1 Foreign Libraries

Just as Guile can load up Scheme libraries at run-time, Guile can also load some system libraries written in C or other low-level languages. We refer to these as dynamically-loadable modules as foreign libraries, to distinguish them from native libraries written in Scheme or other languages implemented by Guile.

Foreign libraries usually come in two forms. Some foreign libraries are part of the operating system, such as the compression library libz. These shared libraries are built in such a way that many programs can use their functionality without duplicating their code. When a program written in C is built, it can declare that it uses a specific set of shared libraries. When the program is run, the operating system takes care of locating and loading the shared libraries.

The operating system components that can dynamically load and link shared libraries when a program is run are also available programmatically during a program’s execution. This is the interface that’s most useful for Guile, and this is what we mean in Guile when we refer to dynamic linking. Dynamic linking at run-time is sometimes called dlopening, to distinguish it from the dynamic linking that happens at program start-up.

The other kind of foreign library is sometimes known as a module, plug-in, bundle, or an extension. These foreign libraries aren’t meant to be linked to by C programs, but rather only to be dynamically loaded at run-time – they extend some main program with functionality, but don’t stand on their own. Sometimes a Guile library will implement some of its functionality in a loadable module.

In either case, the interface on the Guile side is the same. You load the interface using load-foreign-library. The resulting foreign library object implements a simple lookup interface whereby the user can get addresses of data or code exported by the library. There is no facility to inspect foreign libraries; you have to know what’s in there already before you look.

Routines for loading foreign libraries and accessing their contents are implemented in the (system foreign-library) module.

(use-modules (system foreign-library))
Scheme Procedure: load-foreign-library [library] [#:extensions=system-library-extensions] [#:search-ltdl-library-path?=#t] [#:search-path=search-path] [#:search-system-paths?=#t] [#:lazy?=#t] [#:global=#f]

[#:rename-on-cygwin?=#t] Find the shared library denoted by library (a string or #f) and link it into the running Guile application. When everything works out, return a Scheme object suitable for representing the linked object file. Otherwise an error is thrown.

If library argument is omitted, it defaults to #f. If library is false, the resulting foreign library gives access to all symbols available for dynamic linking in the main binary.

It is not necessary to include any extension such as .so in library. For each system, Guile has a default set of extensions that it will try. On GNU systems, the default extension set is just .so; on Windows, just .dll; and on Darwin (Mac OS), it is .bundle, .so, and .dylib. Pass #:extensions extensions to override the default extensions list. If library contains one of the extensions, no extensions are tried, so it is possible to specify the extension if you know exactly what file to load.

Unless library denotes an absolute file name or otherwise contains a directory separator (/, and also \ on Windows), Guile will search for the library in the directories listed in search-paths. The default search path has three components, which can all be overriden by colon-delimited (semicolon on Windows) environment variables:

GUILE_EXTENSIONS_PATH

This is the main environment variable for users to add directories containing Guile extensions. The default value has no entries. This environment variable was added in Guile 3.0.6.

LTDL_LIBRARY_PATH

Before Guile 3.0.6, Guile loaded foreign libraries using libltdl, the dynamic library loader provided by libtool. This loader used LTDL_LIBRARY_PATH, and for backwards compatibility we still support that path.

However, libltdl would not only open .so (or .dll and so on) files, but also the .la files created by libtool. In installed libraries – libraries that are in the target directories of make install.la files are never needed, to the extent that most GNU/Linux distributions remove them entirely. It is sufficient to just load the .so (or .dll and so on) files, which are always located in the same directory as the .la files.

But for uninstalled dynamic libraries, like those in a build tree, the situation is a bit of a mess. If you have a project that uses libtool to build libraries – which is the case for Guile, and for most projects using autotools – and you build foo.so in directory D, libtool will put foo.la in D, but foo.so gets put into D/.libs.

Users were mostly oblivious to this situation, as libltdl had special logic to be able to read the .la file to know where to find the .so, even from an uninstalled build tree, preventing the existence of .libs from leaking out to the user.

We don’t use libltdl now, essentially for flexibility and error-reporting reasons. But, to keep this old use-case working, if search-ltdl-library-path? is true, we add each entry of LTDL_LIBRARY_PATH to the default extensions load path, additionally adding the .libs subdirextories for each entry, in case there are .so files there instead of alongside the .la files.

GUILE_SYSTEM_EXTENSIONS_PATH

The last path in Guile’s search path belongs to Guile itself, and defaults to the libdir and the extensiondir, in that order. For example, if you install to /opt/guile, these would probably be /opt/guile/lib and /opt/guile/lib/guile/3.0/extensions, respectively. See Parallel Installations, for more details on extensionsdir.

Finally, if no library is found in the search path, and if library is not absolute and does not include directory separators, and if search-system-paths? is true, the operating system may have its own logic for where to locate library. For example, on GNU, there will be a default set of paths (often /usr/lib and /lib, though it depends on the system), and the LD_LIBRARY_PATH environment variable can add additional paths. Other operating systems have other conventions.

Falling back to the operating system for search is usually not a great thing; it is a recipe for making programs that work on one machine but not on others. Still, when wrapping system libraries, it can be the only way to get things working at all.

If lazy? is true (the default), Guile will request the operating system to resolve symbols used by the loaded library as they are first used. If global? is true, symbols defined by the loaded library will be available when other modules need to resolve symbols; the default is #f, which keeps symbols local.

If rename-on-cygwin? is true (the default) – on Cygwin hosts only – the search behavior is modified such that a filename that starts with “lib” will be searched for under the name “cyg”, as is customary for Cygwin.

The environment variables mentioned above are parsed when the foreign-library module is first loaded and bound to parameters. Null path components, for example the three components of GUILE_SYSTEM_EXTENSIONS_PATH="::", are ignored.

Scheme Parameter: guile-extensions-path
Scheme Parameter: ltdl-library-path
Scheme Parameter: guile-system-extensions-path

Parameters whose initial values are taken from GUILE_EXTENSIONS_PATH, LTDL_LIBRARY_PATH, and GUILE_SYSTEM_EXTENSIONS_PATH, respectively. See Parameters. The current values of these parameters are used when building the search path when load-foreign-library is called, unless the caller explicitly passes a #:search-path argument.

Scheme Procedure: foreign-library? obj

Return #t if obj is a foreign library, or #f otherwise.