To enhance modularity, similar functions are defined in one source file
(with a .c suffix, see Headers for more). After running
make, each human-readable, .c file is translated (or
compiled) into a computer-readable “object” file (ending with
.o). Note that object files are also created when building programs,
they aren’t particular to libraries. Try opening Gnuastro’s lib/ and
bin/progname/ directories after running
make to see these
object files136. Afterwards, the object files are linked together
to create an executable program or a library.
The object files contain the full definition of the functions in the
respective .c file along with a list of any other function (or
generally “symbol”) that is referenced there. To get a list of those
functions you can use the
nm program which is part of GNU
Binutils. For example from the top Gnuastro directory, run:
$ nm bin/arithmetic/arithmetic.o
This will print a list of all the functions (more generally, ‘symbols’)
that were called within bin/arithmetic/arithmetic.c along with some
further information (for example a
T in the second column shows that
this function is actually defined here,
U says that it is undefined
here). Try opening the .c file to check some of these functions for
your self. Run
info nm for more information.
To recap, the compiler created the separate object files mentioned
above for each .c file. The linker will then combine all the
symbols of the various object files (and libraries) into one program or
library. In the case of Arithmetic (a program) the contents of the object
files in bin/arithmetic/ are copied (and re-ordered) into one final
executable file which we can run from the operating system. When the
symbols (computer-readable function definitions in most cases) are copied
into the output like this, we call the process static linking. Let’s
have a closer look at static linking: we’ll assume you have installed
Gnuastro into the default /usr/local/ directory (see
Installation directory). If you tried the
nm command on one
of Arithmetic’s object files above, then with the command below you can
confirm that all the functions that were defined in the object files (had a
T in the second column) are also defined in the astarithmetic
$ nm /usr/local/bin/astarithmetic
But you will notice that there are still many undefined symbols (have a
U in the second column). One class of such functions are Gnuastro’s
own library functions that start with ‘
$ nm /usr/local/bin/astarithmetic | grep gal_
These undefined symbols (functions) will be linked to the executable every time you run arithmetic. Therefore they are known as dynamically linked libraries 137. When the functions of a library need to be dynamically linked, the library is known as a shared library. As we saw above, static linking is done when the executable is being built. However, when a library is linked dynamically, its symbols are only checked with the available libraries at build time: they are not actually copied into the executable. Every time you run the program, the linker will be activated and will try to link the program to the installed library before it starts. If you want all the libraries to be statically linked to the executables, you have to tell Libtool (which Gnuastro uses for the linking) to disable shared libraries at configure time138:
$ configure --disable-shared
Try configuring, statically building and installing Gnuastro with the
command above. Then check the
gal_ symbols in the installed
Arithmetic executable like before. You will see that they are actually
copied this time (have a
T in the second column). If the second
column doesn’t convince you, look at the executable file size with the
$ ls -lh /usr/local/bin/astarithmetic
It should be around 4.2 Megabytes with this static linking. If you configure and build Gnuastro again with shared libraries enabled (which is the default), you will notice that it is roughly 100 Kilobytes! This huge difference would have been very significant in the old days, but with the roughly Terabyte storage drives commonly in use today, it is negligible. Fortunately, output file size is not the only benefit of dynamic linking: since it links to the libraries at run-time (rather than build-time), you don’t have to re-build a higher-level program or library when an update comes for one of the lower-level libraries it depends on. You just install the new low-level library and it will automatically be used next time in your higher-level tools. To be fair, this also creates a few complications139:
To see a list of all the shared libraries that are needed for a program or
a shared library to run, you can use the GNU C library’s
ldd140 program, for example:
$ ldd /usr/local/bin/astarithmetic
Library file names start with a lib and end with suffix depending on their type as described below. In between these two is the name of the library, for example libgnuastro.a (Gnuastro’s static library) and libgsl.so.0.0.0 (GSL’s shared library).
For each shared library, we also have two symbolic links ending with .so.X and .so. They are automatically set by the installer, but you can change them (point them to another version of the library) when you have multiple versions on your system.
For those libraries that use GNU Libtool (including Gnuastro and its dependencies), both static and dynamic libraries are built and installed in the prefix/lib/ directory (see Installation directory). In this way other programs can make which ever kind of link that they want.
To link with a library, the linker needs to know where to find the library. You do that with two separate options to the linker (see Summary and example on libraries for an example):
Will tell the linker to look into DIR for the libraries. For example -L/usr/local/lib, or -L/home/yourname/.local/lib. You can make multiple calls to this option, so the linker looks into several directories. Note that the space between L and the directory is optional and commonly not used.
Specify the unique name of a library to be linked. As discussed above, library file names have fixed parts which must not be given to this option. So -lgsl will guide the linker to either look for libgsl.a or libgsl.so (depending on the type of linking it is suppose to do). You can link many libraries by repeated calls to this option.
Very important: The place of this option on the command line matters. This is often a source of confusion for beginners, so let’s assume you have asked the linker to link with library A using this option. As soon as the linker confronts this option, it looks into the list of the undefined symbols it has found until that point and does a search in library A for any of those symbols. If any pending undefined symbol is found in library A, it is used. After the search in undefined symbols is complete, the contents of library A are completely discarded from the linker’s memory. Therefore, if a later object file or library uses an unlinked symbol in library A, the linker will abort after it has finished its search in all the input libraries or object files.
As an example, Gnuastro’s
gal_array_dlog10_array function depends on
log10 function of the C Math library (specified with
-lm). So the proper way to link something that uses this function
is -lgnuastro -lm. If instead, you give: -lm -lgnuastro
the linker will complain and abort.
Gnuastro uses GNU Libtool for portable library creation. Libtool will also make a .lo file for each .c file when building libraries (.lo files are human-readable).
Do not confuse dynamically linked libraries with dynamically loaded libraries. The former (that is discussed here) are only loaded once at the program startup. However, the latter can be loaded anytime during the program’s execution, they are also known as plugins.
Libtool is very common and is commonly used. Therefore, you can use this option to configure on most programs using the GNU build system if you want static linking.
Both of these can be avoided by joining the mailing lists of the lower-level libraries and checking the changes in newer versions before installing them. Updates that result in such behaviors are generally heavily emphasized in the release notes.
If your operating system is not using the GNU C library, you might need another tool.