The GNU C Library

Short Contents

Table of Contents

Next: , Previous: (dir), Up: (dir)

Main Menu

This is Edition 0.12, last updated 2007-10-27, of The GNU C Library Reference Manual, for Version 2.8 of the GNU C Library.

Appendices

Indices

--- The Detailed Node Listing ---

Introduction

Standards and Portability

Using the Library

Error Reporting

Memory

Memory Allocation

Unconstrained Allocation

Allocation Debugging

Obstacks

Variable Size Automatic

Locking Pages

Character Handling

String and Array Utilities

Argz and Envz Vectors

Character Set Handling

Restartable multibyte conversion

Non-reentrant Conversion

Generic Charset Conversion

Locales

Locale Information

The Lame Way to Locale Data

Message Translation

Message catalogs a la X/Open

The Uniforum approach

Message catalogs with gettext

Searching and Sorting

Pattern Matching

Globbing

Regular Expressions

Word Expansion

I/O Overview

I/O Concepts

File Names

I/O on Streams

Unreading

Formatted Output

Customizing Printf

Formatted Input

Stream Buffering

Other Kinds of Streams

Custom Streams

Formatted Messages

Low-Level I/O

Stream/Descriptor Precautions

Asynchronous I/O

File Status Flags

File System Interface

Accessing Directories

File Attributes

Pipes and FIFOs

Sockets

Socket Addresses

Local Namespace

Internet Namespace

Host Addresses

Open/Close Sockets

Connections

Transferring Data

Datagrams

Inetd

Socket Options

Low-Level Terminal Interface

Terminal Modes

Special Characters

Pseudo-Terminals

Syslog

Submitting Syslog Messages

Mathematics

Pseudo-Random Numbers

Arithmetic

Floating Point Errors

Arithmetic Functions

Parsing of Numbers

Date and Time

Processor And CPU Time

Calendar Time

Parsing Date and Time

Resource Usage And Limitation

Priority

Traditional Scheduling

Memory Resources

Non-Local Exits

Signal Handling

Concepts of Signals

Standard Signals

Signal Actions

Defining Handlers

Atomic Data Access

Generating Signals

Blocking Signals

Waiting for a Signal

BSD Signal Handling

Program Basics

Program Arguments

Parsing Program Arguments

Environment Variables

Program Termination

Processes

Job Control

Implementing a Shell

Functions for Job Control

Name Service Switch

NSS Configuration File

NSS Module Internals

Extending NSS

Users and Groups

User Accounting Database

User Database

Group Database

Netgroup Database

System Management

Filesystem Handling

Mount Information

System Configuration

Sysconf

Cryptographic Functions

Debugging Support

Language Features

Variadic Functions

How Variadic

Data Type Measurements

Floating Type Macros

Installation

Maintenance

Porting

Next: , Previous: Top, Up: Top

1 Introduction

The C language provides no built-in facilities for performing such common operations as input/output, memory management, string manipulation, and the like. Instead, these facilities are defined in a standard library, which you compile and link with your programs. The GNU C library, described in this document, defines all of the library functions that are specified by the ISO C standard, as well as additional features specific to POSIX and other derivatives of the Unix operating system, and extensions specific to the GNU system.

The purpose of this manual is to tell you how to use the facilities of the GNU library. We have mentioned which features belong to which standards to help you identify things that are potentially non-portable to other systems. But the emphasis in this manual is not on strict portability.

Next: , Up: Introduction

1.1 Getting Started

This manual is written with the assumption that you are at least somewhat familiar with the C programming language and basic programming concepts. Specifically, familiarity with ISO standard C (see ISO C), rather than “traditional” pre-ISO C dialects, is assumed.

The GNU C library includes several header files, each of which provides definitions and declarations for a group of related facilities; this information is used by the C compiler when processing your program. For example, the header file stdio.h declares facilities for performing input and output, and the header file string.h declares string processing utilities. The organization of this manual generally follows the same division as the header files.

If you are reading this manual for the first time, you should read all of the introductory material and skim the remaining chapters. There are a lot of functions in the GNU C library and it's not realistic to expect that you will be able to remember exactly how to use each and every one of them. It's more important to become generally familiar with the kinds of facilities that the library provides, so that when you are writing your programs you can recognize when to make use of library functions, and where in this manual you can find more specific information about them.

Next: , Previous: Getting Started, Up: Introduction

1.2 Standards and Portability

This section discusses the various standards and other sources that the GNU C library is based upon. These sources include the ISO C and POSIX standards, and the System V and Berkeley Unix implementations.

The primary focus of this manual is to tell you how to make effective use of the GNU library facilities. But if you are concerned about making your programs compatible with these standards, or portable to operating systems other than GNU, this can affect how you use the library. This section gives you an overview of these standards, so that you will know what they are when they are mentioned in other parts of the manual.

See Library Summary, for an alphabetical list of the functions and other symbols provided by the library. This list also states which standards each function or symbol comes from.

Next: , Up: Standards and Portability

1.2.1 ISO C

The GNU C library is compatible with the C standard adopted by the American National Standards Institute (ANSI): American National Standard X3.159-1989—“ANSI C” and later by the International Standardization Organization (ISO): ISO/IEC 9899:1990, “Programming languages—C”. We here refer to the standard as ISO C since this is the more general standard in respect of ratification. The header files and library facilities that make up the GNU library are a superset of those specified by the ISO C standard.

If you are concerned about strict adherence to the ISO C standard, you should use the ‘-ansi’ option when you compile your programs with the GNU C compiler. This tells the compiler to define only ISO standard features from the library header files, unless you explicitly ask for additional features. See Feature Test Macros, for information on how to do this.

Being able to restrict the library to include only ISO C features is important because ISO C puts limitations on what names can be defined by the library implementation, and the GNU extensions don't fit these limitations. See Reserved Names, for more information about these restrictions.

This manual does not attempt to give you complete details on the differences between ISO C and older dialects. It gives advice on how to write programs to work portably under multiple C dialects, but does not aim for completeness.

Next: , Previous: ISO C, Up: Standards and Portability

1.2.2 POSIX (The Portable Operating System Interface)

The GNU library is also compatible with the ISO POSIX family of standards, known more formally as the Portable Operating System Interface for Computer Environments (ISO/IEC 9945). They were also published as ANSI/IEEE Std 1003. POSIX is derived mostly from various versions of the Unix operating system.

The library facilities specified by the POSIX standards are a superset of those required by ISO C; POSIX specifies additional features for ISO C functions, as well as specifying new additional functions. In general, the additional requirements and functionality defined by the POSIX standards are aimed at providing lower-level support for a particular kind of operating system environment, rather than general programming language support which can run in many diverse operating system environments.

The GNU C library implements all of the functions specified in ISO/IEC 9945-1:1996, the POSIX System Application Program Interface, commonly referred to as POSIX.1. The primary extensions to the ISO C facilities specified by this standard include file system interface primitives (see File System Interface), device-specific terminal control functions (see Low-Level Terminal Interface), and process control functions (see Processes).

Some facilities from ISO/IEC 9945-2:1993, the POSIX Shell and Utilities standard (POSIX.2) are also implemented in the GNU library. These include utilities for dealing with regular expressions and other pattern matching facilities (see Pattern Matching).

Next: , Previous: POSIX, Up: Standards and Portability

1.2.3 Berkeley Unix

The GNU C library defines facilities from some versions of Unix which are not formally standardized, specifically from the 4.2 BSD, 4.3 BSD, and 4.4 BSD Unix systems (also known as Berkeley Unix) and from SunOS (a popular 4.2 BSD derivative that includes some Unix System V functionality). These systems support most of the ISO C and POSIX facilities, and 4.4 BSD and newer releases of SunOS in fact support them all.

The BSD facilities include symbolic links (see Symbolic Links), the select function (see Waiting for I/O), the BSD signal functions (see BSD Signal Handling), and sockets (see Sockets).

Next: , Previous: Berkeley Unix, Up: Standards and Portability

1.2.4 SVID (The System V Interface Description)

The System V Interface Description (SVID) is a document describing the AT&T Unix System V operating system. It is to some extent a superset of the POSIX standard (see POSIX).

The GNU C library defines most of the facilities required by the SVID that are not also required by the ISO C or POSIX standards, for compatibility with System V Unix and other Unix systems (such as SunOS) which include these facilities. However, many of the more obscure and less generally useful facilities required by the SVID are not included. (In fact, Unix System V itself does not provide them all.)

The supported facilities from System V include the methods for inter-process communication and shared memory, the hsearch and drand48 families of functions, fmtmsg and several of the mathematical functions.

Previous: SVID, Up: Standards and Portability

1.2.5 XPG (The X/Open Portability Guide)

The X/Open Portability Guide, published by the X/Open Company, Ltd., is a more general standard than POSIX. X/Open owns the Unix copyright and the XPG specifies the requirements for systems which are intended to be a Unix system.

The GNU C library complies to the X/Open Portability Guide, Issue 4.2, with all extensions common to XSI (X/Open System Interface) compliant systems and also all X/Open UNIX extensions.

The additions on top of POSIX are mainly derived from functionality available in System V and BSD systems. Some of the really bad mistakes in System V systems were corrected, though. Since fulfilling the XPG standard with the Unix extensions is a precondition for getting the Unix brand chances are good that the functionality is available on commercial systems.

Next: , Previous: Standards and Portability, Up: Introduction

1.3 Using the Library

This section describes some of the practical issues involved in using the GNU C library.

Next: , Up: Using the Library

1.3.1 Header Files

Libraries for use by C programs really consist of two parts: header files that define types and macros and declare variables and functions; and the actual library or archive that contains the definitions of the variables and functions.

(Recall that in C, a declaration merely provides information that a function or variable exists and gives its type. For a function declaration, information about the types of its arguments might be provided as well. The purpose of declarations is to allow the compiler to correctly process references to the declared variables and functions. A definition, on the other hand, actually allocates storage for a variable or says what a function does.) In order to use the facilities in the GNU C library, you should be sure that your program source files include the appropriate header files. This is so that the compiler has declarations of these facilities available and can correctly process references to them. Once your program has been compiled, the linker resolves these references to the actual definitions provided in the archive file.

Header files are included into a program source file by the ‘#include’ preprocessor directive. The C language supports two forms of this directive; the first, 

     #include "header"

is typically used to include a header file header that you write yourself; this would contain definitions and declarations describing the interfaces between the different parts of your particular application. By contrast, 

     #include <file.h>

is typically used to include a header file file.h that contains definitions and declarations for a standard library. This file would normally be installed in a standard place by your system administrator. You should use this second form for the C library header files.

Typically, ‘#include’ directives are placed at the top of the C source file, before any other code. If you begin your source files with some comments explaining what the code in the file does (a good idea), put the ‘#include’ directives immediately afterwards, following the feature test macro definition (see Feature Test Macros).

For more information about the use of header files and ‘#include’ directives, see Header Files.

The GNU C library provides several header files, each of which contains the type and macro definitions and variable and function declarations for a group of related facilities. This means that your programs may need to include several header files, depending on exactly which facilities you are using.

Some library header files include other library header files automatically. However, as a matter of programming style, you should not rely on this; it is better to explicitly include all the header files required for the library facilities you are using. The GNU C library header files have been written in such a way that it doesn't matter if a header file is accidentally included more than once; including a header file a second time has no effect. Likewise, if your program needs to include multiple header files, the order in which they are included doesn't matter.

Compatibility Note: Inclusion of standard header files in any order and any number of times works in any ISO C implementation. However, this has traditionally not been the case in many older C implementations.

Strictly speaking, you don't have to include a header file to use a function it declares; you could declare the function explicitly yourself, according to the specifications in this manual. But it is usually better to include the header file because it may define types and macros that are not otherwise available and because it may define more efficient macro replacements for some functions. It is also a sure way to have the correct declaration.

Next: , Previous: Header Files, Up: Using the Library

1.3.2 Macro Definitions of Functions

If we describe something as a function in this manual, it may have a macro definition as well. This normally has no effect on how your program runs—the macro definition does the same thing as the function would. In particular, macro equivalents for library functions evaluate arguments exactly once, in the same way that a function call would. The main reason for these macro definitions is that sometimes they can produce an inline expansion that is considerably faster than an actual function call.

Taking the address of a library function works even if it is also defined as a macro. This is because, in this context, the name of the function isn't followed by the left parenthesis that is syntactically necessary to recognize a macro call.

You might occasionally want to avoid using the macro definition of a function—perhaps to make your program easier to debug. There are two ways you can do this:

For example, suppose the header file stdlib.h declares a function named abs with 

     extern int abs (int);

and also provides a macro definition for abs. Then, in: 

     #include <stdlib.h>
     int f (int *i) { return abs (++*i); }

the reference to abs might refer to either a macro or a function. On the other hand, in each of the following examples the reference is to a function and not a macro. 

     #include <stdlib.h>
     int g (int *i) { return (abs) (++*i); }
     
     #undef abs
     int h (int *i) { return abs (++*i); }

Since macro definitions that double for a function behave in exactly the same way as the actual function version, there is usually no need for any of these methods. In fact, removing macro definitions usually just makes your program slower.

Next: , Previous: Macro Definitions, Up: Using the Library

1.3.3 Reserved Names

The names of all library types, macros, variables and functions that come from the ISO C standard are reserved unconditionally; your program may not redefine these names. All other library names are reserved if your program explicitly includes the header file that defines or declares them. There are several reasons for these restrictions:

In addition to the names documented in this manual, reserved names include all external identifiers (global functions and variables) that begin with an underscore (‘_’) and all identifiers regardless of use that begin with either two underscores or an underscore followed by a capital letter are reserved names. This is so that the library and header files can define functions, variables, and macros for internal purposes without risk of conflict with names in user programs.

Some additional classes of identifier names are reserved for future extensions to the C language or the POSIX.1 environment. While using these names for your own purposes right now might not cause a problem, they do raise the possibility of conflict with future versions of the C or POSIX standards, so you should avoid these names.

In addition, some individual header files reserve names beyond those that they actually define. You only need to worry about these restrictions if your program includes that particular header file.

Previous: Reserved Names, Up: Using the Library

1.3.4 Feature Test Macros

The exact set of features available when you compile a source file is controlled by which feature test macros you define.

If you compile your programs using ‘gcc -ansi’, you get only the ISO C library features, unless you explicitly request additional features by defining one or more of the feature macros. See GNU CC Command Options, for more information about GCC options.

You should define these macros by using ‘#define’ preprocessor directives at the top of your source code files. These directives must come before any #include of a system header file. It is best to make them the very first thing in the file, preceded only by comments. You could also use the ‘-D’ option to GCC, but it's better if you make the source files indicate their own meaning in a self-contained way.

This system exists to allow the library to conform to multiple standards. Although the different standards are often described as supersets of each other, they are usually incompatible because larger standards require functions with names that smaller ones reserve to the user program. This is not mere pedantry — it has been a problem in practice. For instance, some non-GNU programs define functions named getline that have nothing to do with this library's getline. They would not be compilable if all features were enabled indiscriminately.

This should not be used to verify that a program conforms to a limited standard. It is insufficient for this purpose, as it will not protect you from including header files outside the standard, or relying on semantics undefined within the standard.

— Macro: _POSIX_SOURCE

If you define this macro, then the functionality from the POSIX.1 standard (IEEE Standard 1003.1) is available, as well as all of the ISO C facilities.

The state of _POSIX_SOURCE is irrelevant if you define the macro _POSIX_C_SOURCE to a positive integer.

— Macro: _POSIX_C_SOURCE

Define this macro to a positive integer to control which POSIX functionality is made available. The greater the value of this macro, the more functionality is made available.

If you define this macro to a value greater than or equal to 1, then the functionality from the 1990 edition of the POSIX.1 standard (IEEE Standard 1003.1-1990) is made available.

If you define this macro to a value greater than or equal to 2, then the functionality from the 1992 edition of the POSIX.2 standard (IEEE Standard 1003.2-1992) is made available.

If you define this macro to a value greater than or equal to 199309L, then the functionality from the 1993 edition of the POSIX.1b standard (IEEE Standard 1003.1b-1993) is made available.

Greater values for _POSIX_C_SOURCE will enable future extensions. The POSIX standards process will define these values as necessary, and the GNU C Library should support them some time after they become standardized. The 1996 edition of POSIX.1 (ISO/IEC 9945-1: 1996) states that if you define _POSIX_C_SOURCE to a value greater than or equal to 199506L, then the functionality from the 1996 edition is made available.

— Macro: _BSD_SOURCE

If you define this macro, functionality derived from 4.3 BSD Unix is included as well as the ISO C, POSIX.1, and POSIX.2 material.

Some of the features derived from 4.3 BSD Unix conflict with the corresponding features specified by the POSIX.1 standard. If this macro is defined, the 4.3 BSD definitions take precedence over the POSIX definitions.

Due to the nature of some of the conflicts between 4.3 BSD and POSIX.1, you need to use a special BSD compatibility library when linking programs compiled for BSD compatibility. This is because some functions must be defined in two different ways, one of them in the normal C library, and one of them in the compatibility library. If your program defines _BSD_SOURCE, you must give the option ‘-lbsd-compat’ to the compiler or linker when linking the program, to tell it to find functions in this special compatibility library before looking for them in the normal C library.

— Macro: _SVID_SOURCE

If you define this macro, functionality derived from SVID is included as well as the ISO C, POSIX.1, POSIX.2, and X/Open material.

— Macro: _XOPEN_SOURCE
— Macro: _XOPEN_SOURCE_EXTENDED

If you define this macro, functionality described in the X/Open Portability Guide is included. This is a superset of the POSIX.1 and POSIX.2 functionality and in fact _POSIX_SOURCE and _POSIX_C_SOURCE are automatically defined.

As the unification of all Unices, functionality only available in BSD and SVID is also included.

If the macro _XOPEN_SOURCE_EXTENDED is also defined, even more functionality is available. The extra functions will make all functions available which are necessary for the X/Open Unix brand.

If the macro _XOPEN_SOURCE has the value 500 this includes all functionality described so far plus some new definitions from the Single Unix Specification, version 2.

— Macro: _LARGEFILE_SOURCE

If this macro is defined some extra functions are available which rectify a few shortcomings in all previous standards. Specifically, the functions fseeko and ftello are available. Without these functions the difference between the ISO C interface (fseek, ftell) and the low-level POSIX interface (lseek) would lead to problems.

This macro was introduced as part of the Large File Support extension (LFS).

— Macro: _LARGEFILE64_SOURCE

If you define this macro an additional set of functions is made available which enables 32 bit systems to use files of sizes beyond the usual limit of 2GB. This interface is not available if the system does not support files that large. On systems where the natural file size limit is greater than 2GB (i.e., on 64 bit systems) the new functions are identical to the replaced functions.

The new functionality is made available by a new set of types and functions which replace the existing ones. The names of these new objects contain 64 to indicate the intention, e.g., off_t vs. off64_t and fseeko vs. fseeko64.

This macro was introduced as part of the Large File Support extension (LFS). It is a transition interface for the period when 64 bit offsets are not generally used (see _FILE_OFFSET_BITS).

— Macro: _FILE_OFFSET_BITS

This macro determines which file system interface shall be used, one replacing the other. Whereas _LARGEFILE64_SOURCE makes the 64 bit interface available as an additional interface, _FILE_OFFSET_BITS allows the 64 bit interface to replace the old interface.

If _FILE_OFFSET_BITS is undefined, or if it is defined to the value 32, nothing changes. The 32 bit interface is used and types like off_t have a size of 32 bits on 32 bit systems.

If the macro is defined to the value 64, the large file interface replaces the old interface. I.e., the functions are not made available under different names (as they are with _LARGEFILE64_SOURCE). Instead the old function names now reference the new functions, e.g., a call to fseeko now indeed calls fseeko64.

This macro should only be selected if the system provides mechanisms for handling large files. On 64 bit systems this macro has no effect since the *64 functions are identical to the normal functions.

This macro was introduced as part of the Large File Support extension (LFS).

— Macro: _ISOC99_SOURCE

Until the revised ISO C standard is widely adopted the new features are not automatically enabled. The GNU libc nevertheless has a complete implementation of the new standard and to enable the new features the macro _ISOC99_SOURCE should be defined.

— Macro: _GNU_SOURCE

If you define this macro, everything is included: ISO C89, ISO C99, POSIX.1, POSIX.2, BSD, SVID, X/Open, LFS, and GNU extensions. In the cases where POSIX.1 conflicts with BSD, the POSIX definitions take precedence.

If you want to get the full effect of _GNU_SOURCE but make the BSD definitions take precedence over the POSIX definitions, use this sequence of definitions: 

          #define _GNU_SOURCE
          #define _BSD_SOURCE
          #define _SVID_SOURCE

Note that if you do this, you must link your program with the BSD compatibility library by passing the ‘-lbsd-compat’ option to the compiler or linker. NB: If you forget to do this, you may get very strange errors at run time.

— Macro: _REENTRANT
— Macro: _THREAD_SAFE

If you define one of these macros, reentrant versions of several functions get declared. Some of the functions are specified in POSIX.1c but many others are only available on a few other systems or are unique to GNU libc. The problem is the delay in the standardization of the thread safe C library interface.

Unlike on some other systems, no special version of the C library must be used for linking. There is only one version but while compiling this it must have been specified to compile as thread safe.

We recommend you use _GNU_SOURCE in new programs. If you don't specify the ‘-ansi’ option to GCC and don't define any of these macros explicitly, the effect is the same as defining _POSIX_C_SOURCE to 2 and _POSIX_SOURCE, _SVID_SOURCE, and _BSD_SOURCE to 1.

When you define a feature test macro to request a larger class of features, it is harmless to define in addition a feature test macro for a subset of those features. For example, if you define _POSIX_C_SOURCE, then defining _POSIX_SOURCE as well has no effect. Likewise, if you define _GNU_SOURCE, then defining either _POSIX_SOURCE or _POSIX_C_SOURCE or _SVID_SOURCE as well has no effect.

Note, however, that the features of _BSD_SOURCE are not a subset of any of the other feature test macros supported. This is because it defines BSD features that take precedence over the POSIX features that are requested by the other macros. For this reason, defining _BSD_SOURCE in addition to the other feature test macros does have an effect: it causes the BSD features to take priority over the conflicting POSIX features.

Previous: Using the Library, Up: Introduction

1.4 Roadmap to the Manual

Here is an overview of the contents of the remaining chapters of this manual.

If you already know the name of the facility you are interested in, you can look it up in Library Summary. This gives you a summary of its syntax and a pointer to where you can find a more detailed description. This appendix is particularly useful if you just want to verify the order and type of arguments to a function, for example. It also tells you what standard or system each function, variable, or macro is derived from.

Next: , Previous: Introduction, Up: Top

2 Error Reporting

Many functions in the GNU C library detect and report error conditions, and sometimes your programs need to check for these error conditions. For example, when you open an input file, you should verify that the file was actually opened correctly, and print an error message or take other appropriate action if the call to the library function failed.

This chapter describes how the error reporting facility works. Your program should include the header file errno.h to use this facility.

Next: , Up: Error Reporting

2.1 Checking for Errors

Most library functions return a special value to indicate that they have failed. The special value is typically -1, a null pointer, or a constant such as EOF that is defined for that purpose. But this return value tells you only that an error has occurred. To find out what kind of error it was, you need to look at the error code stored in the variable errno. This variable is declared in the header file errno.h.

— Variable: volatile int errno

The variable errno contains the system error number. You can change the value of errno.

Since errno is declared volatile, it might be changed asynchronously by a signal handler; see Defining Handlers. However, a properly written signal handler saves and restores the value of errno, so you generally do not need to worry about this possibility except when writing signal handlers.

The initial value of errno at program startup is zero. Many library functions are guaranteed to set it to certain nonzero values when they encounter certain kinds of errors. These error conditions are listed for each function. These functions do not change errno when they succeed; thus, the value of errno after a successful call is not necessarily zero, and you should not use errno to determine whether a call failed. The proper way to do that is documented for each function. If the call failed, you can examine errno.

Many library functions can set errno to a nonzero value as a result of calling other library functions which might fail. You should assume that any library function might alter errno when the function returns an error.

Portability Note: ISO C specifies errno as a “modifiable lvalue” rather than as a variable, permitting it to be implemented as a macro. For example, its expansion might involve a function call, like *_errno (). In fact, that is what it is on the GNU system itself. The GNU library, on non-GNU systems, does whatever is right for the particular system.

There are a few library functions, like sqrt and atan, that return a perfectly legitimate value in case of an error, but also set errno. For these functions, if you want to check to see whether an error occurred, the recommended method is to set errno to zero before calling the function, and then check its value afterward.

All the error codes have symbolic names; they are macros defined in errno.h. The names start with ‘E’ and an upper-case letter or digit; you should consider names of this form to be reserved names. See Reserved Names.

The error code values are all positive integers and are all distinct, with one exception: EWOULDBLOCK and EAGAIN are the same. Since the values are distinct, you can use them as labels in a switch statement; just don't use both EWOULDBLOCK and EAGAIN. Your program should not make any other assumptions about the specific values of these symbolic constants.

The value of errno doesn't necessarily have to correspond to any of these macros, since some library functions might return other error codes of their own for other situations. The only values that are guaranteed to be meaningful for a particular library function are the ones that this manual lists for that function.

On non-GNU systems, almost any system call can return EFAULT if it is given an invalid pointer as an argument. Since this could only happen as a result of a bug in your program, and since it will not happen on the GNU system, we have saved space by not mentioning EFAULT in the descriptions of individual functions.

In some Unix systems, many system calls can also return EFAULT if given as an argument a pointer into the stack, and the kernel for some obscure reason fails in its attempt to extend the stack. If this ever happens, you should probably try using statically or dynamically allocated memory instead of stack memory on that system.

Next: , Previous: Checking for Errors, Up: Error Reporting

2.2 Error Codes

The error code macros are defined in the header file errno.h. All of them expand into integer constant values. Some of these error codes can't occur on the GNU system, but they can occur using the GNU library on other systems.

— Macro: int EPERM

Operation not permitted; only the owner of the file (or other resource) or processes with special privileges can perform the operation.

— Macro: int ENOENT

No such file or directory. This is a “file doesn't exist” error for ordinary files that are referenced in contexts where they are expected to already exist.

— Macro: int ESRCH

No process matches the specified process ID.

— Macro: int EINTR

Interrupted function call; an asynchronous signal occurred and prevented completion of the call. When this happens, you should try the call again.

You can choose to have functions resume after a signal that is handled, rather than failing with EINTR; see Interrupted Primitives.

— Macro: int EIO

Input/output error; usually used for physical read or write errors.

— Macro: int ENXIO

No such device or address. The system tried to use the device represented by a file you specified, and it couldn't find the device. This can mean that the device file was installed incorrectly, or that the physical device is missing or not correctly attached to the computer.

— Macro: int E2BIG

Argument list too long; used when the arguments passed to a new program being executed with one of the exec functions (see Executing a File) occupy too much memory space. This condition never arises in the GNU system.

— Macro: int ENOEXEC

Invalid executable file format. This condition is detected by the exec functions; see Executing a File.

— Macro: int EBADF

Bad file descriptor; for example, I/O on a descriptor that has been closed or reading from a descriptor open only for writing (or vice versa).

— Macro: int ECHILD

There are no child processes. This error happens on operations that are supposed to manipulate child processes, when there aren't any processes to manipulate.

— Macro: int EDEADLK

Deadlock avoided; allocating a system resource would have resulted in a deadlock situation. The system does not guarantee that it will notice all such situations. This error means you got lucky and the system noticed; it might just hang. See File Locks, for an example.

— Macro: int ENOMEM

No memory available. The system cannot allocate more virtual memory because its capacity is full.

— Macro: int EACCES

Permission denied; the file permissions do not allow the attempted operation.

— Macro: int EFAULT

Bad address; an invalid pointer was detected. In the GNU system, this error never happens; you get a signal instead.

— Macro: int ENOTBLK

A file that isn't a block special file was given in a situation that requires one. For example, trying to mount an ordinary file as a file system in Unix gives this error.

— Macro: int EBUSY

Resource busy; a system resource that can't be shared is already in use. For example, if you try to delete a file that is the root of a currently mounted filesystem, you get this error.

— Macro: int EEXIST

File exists; an existing file was specified in a context where it only makes sense to specify a new file.

— Macro: int EXDEV

An attempt to make an improper link across file systems was detected. This happens not only when you use link (see Hard Links) but also when you rename a file with rename (see Renaming Files).

— Macro: int ENODEV

The wrong type of device was given to a function that expects a particular sort of device.

— Macro: int ENOTDIR

A file that isn't a directory was specified when a directory is required.

— Macro: int EISDIR

File is a directory; you cannot open a directory for writing, or create or remove hard links to it.

— Macro: int EINVAL

Invalid argument. This is used to indicate various kinds of problems with passing the wrong argument to a library function.

— Macro: int EMFILE

The current process has too many files open and can't open any more. Duplicate descriptors do count toward this limit.

In BSD and GNU, the number of open files is controlled by a resource limit that can usually be increased. If you get this error, you might want to increase the RLIMIT_NOFILE limit or make it unlimited; see Limits on Resources.

— Macro: int ENFILE

There are too many distinct file openings in the entire system. Note that any number of linked channels count as just one file opening; see Linked Channels. This error never occurs in the GNU system.

— Macro: int ENOTTY

Inappropriate I/O control operation, such as trying to set terminal modes on an ordinary file.

— Macro: int ETXTBSY

An attempt to execute a file that is currently open for writing, or write to a file that is currently being executed. Often using a debugger to run a program is considered having it open for writing and will cause this error. (The name stands for “text file busy”.) This is not an error in the GNU system; the text is copied as necessary.

— Macro: int EFBIG

File too big; the size of a file would be larger than allowed by the system.

— Macro: int ENOSPC

No space left on device; write operation on a file failed because the disk is full.

— Macro: int ESPIPE

Invalid seek operation (such as on a pipe).

— Macro: int EROFS

An attempt was made to modify something on a read-only file system.

— Macro: int EMLINK

Too many links; the link count of a single file would become too large. rename can cause this error if the file being renamed already has as many links as it can take (see Renaming Files).

— Macro: int EPIPE

Broken pipe; there is no process reading from the other end of a pipe. Every library function that returns this error code also generates a SIGPIPE signal; this signal terminates the program if not handled or blocked. Thus, your program will never actually see EPIPE unless it has handled or blocked SIGPIPE.

— Macro: int EDOM

Domain error; used by mathematical functions when an argument value does not fall into the domain over which the function is defined.

— Macro: int ERANGE

Range error; used by mathematical functions when the result value is not representable because of overflow or underflow.

— Macro: int EAGAIN

Resource temporarily unavailable; the call might work if you try again later. The macro EWOULDBLOCK is another name for EAGAIN; they are always the same in the GNU C library.

This error can happen in a few different situations:

— Macro: int EWOULDBLOCK

In the GNU C library, this is another name for EAGAIN (above). The values are always the same, on every operating system.

C libraries in many older Unix systems have EWOULDBLOCK as a separate error code.

— Macro: int EINPROGRESS

An operation that cannot complete immediately was initiated on an object that has non-blocking mode selected. Some functions that must always block (such as connect; see Connecting) never return EAGAIN. Instead, they return EINPROGRESS to indicate that the operation has begun and will take some time. Attempts to manipulate the object before the call completes return EALREADY. You can use the select function to find out when the pending operation has completed; see Waiting for I/O.

— Macro: int EALREADY

An operation is already in progress on an object that has non-blocking mode selected.

— Macro: int ENOTSOCK

A file that isn't a socket was specified when a socket is required.

— Macro: int EMSGSIZE

The size of a message sent on a socket was larger than the supported maximum size.

— Macro: int EPROTOTYPE

The socket type does not support the requested communications protocol.

— Macro: int ENOPROTOOPT

You specified a socket option that doesn't make sense for the particular protocol being used by the socket. See Socket Options.

— Macro: int EPROTONOSUPPORT

The socket domain does not support the requested communications protocol (perhaps because the requested protocol is completely invalid). See Creating a Socket.

— Macro: int ESOCKTNOSUPPORT

The socket type is not supported.

— Macro: int EOPNOTSUPP

The operation you requested is not supported. Some socket functions don't make sense for all types of sockets, and others may not be implemented for all communications protocols. In the GNU system, this error can happen for many calls when the object does not support the particular operation; it is a generic indication that the server knows nothing to do for that call.

— Macro: int EPFNOSUPPORT

The socket communications protocol family you requested is not supported.

— Macro: int EAFNOSUPPORT

The address family specified for a socket is not supported; it is inconsistent with the protocol being used on the socket. See Sockets.

— Macro: int EADDRINUSE

The requested socket address is already in use. See Socket Addresses.

— Macro: int EADDRNOTAVAIL

The requested socket address is not available; for example, you tried to give a socket a name that doesn't match the local host name. See Socket Addresses.

— Macro: int ENETDOWN

A socket operation failed because the network was down.

— Macro: int ENETUNREACH

A socket operation failed because the subnet containing the remote host was unreachable.

— Macro: int ENETRESET

A network connection was reset because the remote host crashed.

— Macro: int ECONNABORTED

A network connection was aborted locally.

— Macro: int ECONNRESET

A network connection was closed for reasons outside the control of the local host, such as by the remote machine rebooting or an unrecoverable protocol violation.

— Macro: int ENOBUFS

The kernel's buffers for I/O operations are all in use. In GNU, this error is always synonymous with ENOMEM; you may get one or the other from network operations.

— Macro: int EISCONN

You tried to connect a socket that is already connected. See Connecting.

— Macro: int ENOTCONN

The socket is not connected to anything. You get this error when you try to transmit data over a socket, without first specifying a destination for the data. For a connectionless socket (for datagram protocols, such as UDP), you get EDESTADDRREQ instead.

— Macro: int EDESTADDRREQ

No default destination address was set for the socket. You get this error when you try to transmit data over a connectionless socket, without first specifying a destination for the data with connect.

— Macro: int ESHUTDOWN

The socket has already been shut down.

— Macro: int ETOOMANYREFS

???

— Macro: int ETIMEDOUT

A socket operation with a specified timeout received no response during the timeout period.

— Macro: int ECONNREFUSED

A remote host refused to allow the network connection (typically because it is not running the requested service).

— Macro: int ELOOP

Too many levels of symbolic links were encountered in looking up a file name. This often indicates a cycle of symbolic links.

— Macro: int ENAMETOOLONG

Filename too long (longer than PATH_MAX; see Limits for Files) or host name too long (in gethostname or sethostname; see Host Identification).

— Macro: int EHOSTDOWN

The remote host for a requested network connection is down.

— Macro: int EHOSTUNREACH

The remote host for a requested network connection is not reachable.

— Macro: int ENOTEMPTY

Directory not empty, where an empty directory was expected. Typically, this error occurs when you are trying to delete a directory.

— Macro: int EPROCLIM

This means that the per-user limit on new process would be exceeded by an attempted fork. See Limits on Resources, for details on the RLIMIT_NPROC limit.

— Macro: int EUSERS

The file quota system is confused because there are too many users.

— Macro: int EDQUOT

The user's disk quota was exceeded.

— Macro: int ESTALE

Stale NFS file handle. This indicates an internal confusion in the NFS system which is due to file system rearrangements on the server host. Repairing this condition usually requires unmounting and remounting the NFS file system on the local host.

— Macro: int EREMOTE

An attempt was made to NFS-mount a remote file system with a file name that already specifies an NFS-mounted file. (This is an error on some operating systems, but we expect it to work properly on the GNU system, making this error code impossible.)

— Macro: int EBADRPC

???

— Macro: int ERPCMISMATCH

???

— Macro: int EPROGUNAVAIL

???

— Macro: int EPROGMISMATCH

???

— Macro: int EPROCUNAVAIL

???

— Macro: int ENOLCK

No locks available. This is used by the file locking facilities; see File Locks. This error is never generated by the GNU system, but it can result from an operation to an NFS server running another operating system.

— Macro: int EFTYPE

Inappropriate file type or format. The file was the wrong type for the operation, or a data file had the wrong format.

On some systems chmod returns this error if you try to set the sticky bit on a non-directory file; see Setting Permissions.

— Macro: int EAUTH

???

— Macro: int ENEEDAUTH

???

— Macro: int ENOSYS

Function not implemented. This indicates that the function called is not implemented at all, either in the C library itself or in the operating system. When you get this error, you can be sure that this particular function will always fail with ENOSYS unless you install a new version of the C library or the operating system.

— Macro: int ENOTSUP

Not supported. A function returns this error when certain parameter values are valid, but the functionality they request is not available. This can mean that the function does not implement a particular command or option value or flag bit at all. For functions that operate on some object given in a parameter, such as a file descriptor or a port, it might instead mean that only that specific object (file descriptor, port, etc.) is unable to support the other parameters given; different file descriptors might support different ranges of parameter values.

If the entire function is not available at all in the implementation, it returns ENOSYS instead.

— Macro: int EILSEQ

While decoding a multibyte character the function came along an invalid or an incomplete sequence of bytes or the given wide character is invalid.

— Macro: int EBACKGROUND

In the GNU system, servers supporting the term protocol return this error for certain operations when the caller is not in the foreground process group of the terminal. Users do not usually see this error because functions such as read and write translate it into a SIGTTIN or SIGTTOU signal. See Job Control, for information on process groups and these signals.

— Macro: int EDIED

In the GNU system, opening a file returns this error when the file is translated by a program and the translator program dies while starting up, before it has connected to the file.

— Macro: int ED

The experienced user will know what is wrong.

— Macro: int EGREGIOUS

You did what?

— Macro: int EIEIO

Go home and have a glass of warm, dairy-fresh milk.

— Macro: int EGRATUITOUS

This error code has no purpose.

— Macro: int EBADMSG
— Macro: int EIDRM
— Macro: int EMULTIHOP
— Macro: int ENODATA
— Macro: int ENOLINK
— Macro: int ENOMSG
— Macro: int ENOSR
— Macro: int ENOSTR
— Macro: int EOVERFLOW
— Macro: int EPROTO
— Macro: int ETIME
— Macro: int ECANCELED

Operation canceled; an asynchronous operation was canceled before it completed. See Asynchronous I/O. When you call aio_cancel, the normal result is for the operations affected to complete with this error; see Cancel AIO Operations.

The following error codes are defined by the Linux/i386 kernel. They are not yet documented.

— Macro: int ERESTART
— Macro: int ECHRNG
— Macro: int EL2NSYNC
— Macro: int EL3HLT
— Macro: int EL3RST
— Macro: int ELNRNG
— Macro: int EUNATCH
— Macro: int ENOCSI
— Macro: int EL2HLT
— Macro: int EBADE
— Macro: int EBADR
— Macro: int EXFULL
— Macro: int ENOANO
— Macro: int EBADRQC
— Macro: int EBADSLT
— Macro: int EDEADLOCK
— Macro: int EBFONT
— Macro: int ENONET
— Macro: int ENOPKG
— Macro: int EADV
— Macro: int ESRMNT
— Macro: int ECOMM
— Macro: int EDOTDOT
— Macro: int ENOTUNIQ
— Macro: int EBADFD
— Macro: int EREMCHG
— Macro: int ELIBACC
— Macro: int ELIBBAD
— Macro: int ELIBSCN
— Macro: int ELIBMAX
— Macro: int ELIBEXEC
— Macro: int ESTRPIPE
— Macro: int EUCLEAN
— Macro: int ENOTNAM
— Macro: int ENAVAIL
— Macro: int EISNAM
— Macro: int EREMOTEIO
— Macro: int ENOMEDIUM
— Macro: int EMEDIUMTYPE
— Macro: int ENOKEY
— Macro: int EKEYEXPIRED
— Macro: int EKEYREVOKED
— Macro: int EKEYREJECTED
— Macro: int EOWNERDEAD
— Macro: int ENOTRECOVERABLE

Previous: Error Codes, Up: Error Reporting

2.3 Error Messages

The library has functions and variables designed to make it easy for your program to report informative error messages in the customary format about the failure of a library call. The functions strerror and perror give you the standard error message for a given error code; the variable program_invocation_short_name gives you convenient access to the name of the program that encountered the error.

— Function: char * strerror (int errnum)

The strerror function maps the error code (see Checking for Errors) specified by the errnum argument to a descriptive error message string. The return value is a pointer to this string.

The value errnum normally comes from the variable errno.

You should not modify the string returned by strerror. Also, if you make subsequent calls to strerror, the string might be overwritten. (But it's guaranteed that no library function ever calls strerror behind your back.)

The function strerror is declared in string.h.

— Function: char * strerror_r (int errnum, char *buf, size_t n)

The strerror_r function works like strerror but instead of returning the error message in a statically allocated buffer shared by all threads in the process, it returns a private copy for the thread. This might be either some permanent global data or a message string in the user supplied buffer starting at buf with the length of n bytes.

At most n characters are written (including the NUL byte) so it is up to the user to select the buffer large enough.

This function should always be used in multi-threaded programs since there is no way to guarantee the string returned by strerror really belongs to the last call of the current thread.

This function strerror_r is a GNU extension and it is declared in string.h.

— Function: void perror (const char *message)

This function prints an error message to the stream stderr; see Standard Streams. The orientation of stderr is not changed.

If you call perror with a message that is either a null pointer or an empty string, perror just prints the error message corresponding to errno, adding a trailing newline.

If you supply a non-null message argument, then perror prefixes its output with this string. It adds a colon and a space character to separate the message from the error string corresponding to errno.

The function perror is declared in stdio.h.

strerror and perror produce the exact same message for any given error code; the precise text varies from system to system. On the GNU system, the messages are fairly short; there are no multi-line messages or embedded newlines. Each error message begins with a capital letter and does not include any terminating punctuation.

Compatibility Note: The strerror function was introduced in ISO C89. Many older C systems do not support this function yet.

Many programs that don't read input from the terminal are designed to exit if any system call fails. By convention, the error message from such a program should start with the program's name, sans directories. You can find that name in the variable program_invocation_short_name; the full file name is stored the variable program_invocation_name.

— Variable: char * program_invocation_name

This variable's value is the name that was used to invoke the program running in the current process. It is the same as argv[0]. Note that this is not necessarily a useful file name; often it contains no directory names. See Program Arguments.

— Variable: char * program_invocation_short_name

This variable's value is the name that was used to invoke the program running in the current process, with directory names removed. (That is to say, it is the same as program_invocation_name minus everything up to the last slash, if any.)

The library initialization code sets up both of these variables before calling main.

Portability Note: These two variables are GNU extensions. If you want your program to work with non-GNU libraries, you must save the value of argv[0] in main, and then strip off the directory names yourself. We added these extensions to make it possible to write self-contained error-reporting subroutines that require no explicit cooperation from main.

Here is an example showing how to handle failure to open a file correctly. The function open_sesame tries to open the named file for reading and returns a stream if successful. The fopen library function returns a null pointer if it couldn't open the file for some reason. In that situation, open_sesame constructs an appropriate error message using the strerror function, and terminates the program. If we were going to make some other library calls before passing the error code to strerror, we'd have to save it in a local variable instead, because those other library functions might overwrite errno in the meantime. 

     #include <errno.h>
     #include <stdio.h>
     #include <stdlib.h>
     #include <string.h>
     
     FILE *
     open_sesame (char *name)
     {
       FILE *stream;
     
       errno = 0;
       stream = fopen (name, "r");
       if (stream == NULL)
         {
           fprintf (stderr, "%s: Couldn't open file %s; %s\n",
                    program_invocation_short_name, name, strerror (errno));
           exit (EXIT_FAILURE);
         }
       else
         return stream;
     }

Using perror has the advantage that the function is portable and available on all systems implementing ISO C. But often the text perror generates is not what is wanted and there is no way to extend or change what perror does. The GNU coding standard, for instance, requires error messages to be preceded by the program name and programs which read some input files should should provide information about the input file name and the line number in case an error is encountered while reading the file. For these occasions there are two functions available which are widely used throughout the GNU project. These functions are declared in error.h.

— Function: void error (int status, int errnum, const char *format, ...)

The error function can be used to report general problems during program execution. The format argument is a format string just like those given to the printf family of functions. The arguments required for the format can follow the format parameter. Just like perror, error also can report an error code in textual form. But unlike perror the error value is explicitly passed to the function in the errnum parameter. This eliminates the problem mentioned above that the error reporting function must be called immediately after the function causing the error since otherwise errno might have a different value.

The error prints first the program name. If the application defined a global variable error_print_progname and points it to a function this function will be called to print the program name. Otherwise the string from the global variable program_name is used. The program name is followed by a colon and a space which in turn is followed by the output produced by the format string. If the errnum parameter is non-zero the format string output is followed by a colon and a space, followed by the error message for the error code errnum. In any case is the output terminated with a newline.

The output is directed to the stderr stream. If the stderr wasn't oriented before the call it will be narrow-oriented afterwards.

The function will return unless the status parameter has a non-zero value. In this case the function will call exit with the status value for its parameter and therefore never return. If error returns the global variable error_message_count is incremented by one to keep track of the number of errors reported.

— Function: void error_at_line (int status, int errnum, const char *fname, unsigned int lineno, const char *format, ...)

The error_at_line function is very similar to the error function. The only difference are the additional parameters fname and lineno. The handling of the other parameters is identical to that of error except that between the program name and the string generated by the format string additional text is inserted.

Directly following the program name a colon, followed by the file name pointer to by fname, another colon, and a value of lineno is printed.

This additional output of course is meant to be used to locate an error in an input file (like a programming language source code file etc).

If the global variable error_one_per_line is set to a non-zero value error_at_line will avoid printing consecutive messages for the same file and line. Repetition which are not directly following each other are not caught.

Just like error this function only returned if status is zero. Otherwise exit is called with the non-zero value. If error returns the global variable error_message_count is incremented by one to keep track of the number of errors reported.

As mentioned above the error and error_at_line functions can be customized by defining a variable named error_print_progname.

— Variable: void (*) error_print_progname (void)

If the error_print_progname variable is defined to a non-zero value the function pointed to is called by error or error_at_line. It is expected to print the program name or do something similarly useful.

The function is expected to be print to the stderr stream and must be able to handle whatever orientation the stream has.

The variable is global and shared by all threads.

— Variable: unsigned int error_message_count

The error_message_count variable is incremented whenever one of the functions error or error_at_line returns. The variable is global and shared by all threads.

— Variable: int error_one_per_line

The error_one_per_line variable influences only error_at_line. Normally the error_at_line function creates output for every invocation. If error_one_per_line is set to a non-zero value error_at_line keeps track of the last file name and line number for which an error was reported and avoid directly following messages for the same file and line. This variable is global and shared by all threads.

A program which read some input file and reports errors in it could look like this: 

     {
       char *line = NULL;
       size_t len = 0;
       unsigned int lineno = 0;
     
       error_message_count = 0;
       while (! feof_unlocked (fp))
         {
           ssize_t n = getline (&line, &len, fp);
           if (n <= 0)
             /* End of file or error.  */
             break;
           ++lineno;
     
           /* Process the line.  */
           ...
     
           if (Detect error in line)
             error_at_line (0, errval, filename, lineno,
                            "some error text %s", some_variable);
         }
     
       if (error_message_count != 0)
         error (EXIT_FAILURE, 0, "%u errors found", error_message_count);
     }

error and error_at_line are clearly the functions of choice and enable the programmer to write applications which follow the GNU coding standard. The GNU libc additionally contains functions which are used in BSD for the same purpose. These functions are declared in err.h. It is generally advised to not use these functions. They are included only for compatibility.

— Function: void warn (const char *format, ...)

The warn function is roughly equivalent to a call like 

            error (0, errno, format, the parameters)

except that the global variables error respects and modifies are not used.

— Function: void vwarn (const char *format, va_list)

The vwarn function is just like warn except that the parameters for the handling of the format string format are passed in as an value of type va_list.

— Function: void warnx (const char *format, ...)

The warnx function is roughly equivalent to a call like 

            error (0, 0, format, the parameters)

except that the global variables error respects and modifies are not used. The difference to warn is that no error number string is printed.

— Function: void vwarnx (const char *format, va_list)

The vwarnx function is just like warnx except that the parameters for the handling of the format string format are passed in as an value of type va_list.

— Function: void err (int status, const char *format, ...)

The err function is roughly equivalent to a call like 

            error (status, errno, format, the parameters)

except that the global variables error respects and modifies are not used and that the program is exited even if status is zero.

— Function: void verr (int status, const char *format, va_list)

The verr function is just like err except that the parameters for the handling of the format string format are passed in as an value of type va_list.

— Function: void errx (int status, const char *format, ...)

The errx function is roughly equivalent to a call like 

            error (status, 0, format, the parameters)

except that the global variables error respects and modifies are not used and that the program is exited even if status is zero. The difference to err is that no error number string is printed.

— Function: void verrx (int status, const char *format, va_list)

The verrx function is just like errx except that the parameters for the handling of the format string format are passed in as an value of type va_list.

Next: , Previous: Error Reporting, Up: Top

3 Virtual Memory Allocation And Paging

This chapter describes how processes manage and use memory in a system that uses the GNU C library.

The GNU C Library has several functions for dynamically allocating virtual memory in various ways. They vary in generality and in efficiency. The library also provides functions for controlling paging and allocation of real memory.

Memory mapped I/O is not discussed in this chapter. See Memory-mapped I/O.

Next: , Up: Memory

3.1 Process Memory Concepts

One of the most basic resources a process has available to it is memory. There are a lot of different ways systems organize memory, but in a typical one, each process has one linear virtual address space, with addresses running from zero to some huge maximum. It need not be contiguous; i.e., not all of these addresses actually can be used to store data.

The virtual memory is divided into pages (4 kilobytes is typical). Backing each page of virtual memory is a page of real memory (called a frame) or some secondary storage, usually disk space. The disk space might be swap space or just some ordinary disk file. Actually, a page of all zeroes sometimes has nothing at all backing it – there's just a flag saying it is all zeroes. The same frame of real memory or backing store can back multiple virtual pages belonging to multiple processes. This is normally the case, for example, with virtual memory occupied by GNU C library code. The same real memory frame containing the printf function backs a virtual memory page in each of the existing processes that has a printf call in its program.

In order for a program to access any part of a virtual page, the page must at that moment be backed by (“connected to”) a real frame. But because there is usually a lot more virtual memory than real memory, the pages must move back and forth between real memory and backing store regularly, coming into real memory when a process needs to access them and then retreating to backing store when not needed anymore. This movement is called paging.

When a program attempts to access a page which is not at that moment backed by real memory, this is known as a page fault. When a page fault occurs, the kernel suspends the process, places the page into a real page frame (this is called “paging in” or “faulting in”), then resumes the process so that from the process' point of view, the page was in real memory all along. In fact, to the process, all pages always seem to be in real memory. Except for one thing: the elapsed execution time of an instruction that would normally be a few nanoseconds is suddenly much, much, longer (because the kernel normally has to do I/O to complete the page-in). For programs sensitive to that, the functions described in Locking Pages can control it. Within each virtual address space, a process has to keep track of what is at which addresses, and that process is called memory allocation. Allocation usually brings to mind meting out scarce resources, but in the case of virtual memory, that's not a major goal, because there is generally much more of it than anyone needs. Memory allocation within a process is mainly just a matter of making sure that the same byte of memory isn't used to store two different things.

Processes allocate memory in two major ways: by exec and programmatically. Actually, forking is a third way, but it's not very interesting. See Creating a Process.

Exec is the operation of creating a virtual address space for a process, loading its basic program into it, and executing the program. It is done by the “exec” family of functions (e.g. execl). The operation takes a program file (an executable), it allocates space to load all the data in the executable, loads it, and transfers control to it. That data is most notably the instructions of the program (the text), but also literals and constants in the program and even some variables: C variables with the static storage class (see Memory Allocation and C). Once that program begins to execute, it uses programmatic allocation to gain additional memory. In a C program with the GNU C library, there are two kinds of programmatic allocation: automatic and dynamic. See Memory Allocation and C.

Memory-mapped I/O is another form of dynamic virtual memory allocation. Mapping memory to a file means declaring that the contents of certain range of a process' addresses shall be identical to the contents of a specified regular file. The system makes the virtual memory initially contain the contents of the file, and if you modify the memory, the system writes the same modification to the file. Note that due to the magic of virtual memory and page faults, there is no reason for the system to do I/O to read the file, or allocate real memory for its contents, until the program accesses the virtual memory. See Memory-mapped I/O. Just as it programmatically allocates memory, the program can programmatically deallocate (free) it. You can't free the memory that was allocated by exec. When the program exits or execs, you might say that all its memory gets freed, but since in both cases the address space ceases to exist, the point is really moot. See Program Termination. A process' virtual address space is divided into segments. A segment is a contiguous range of virtual addresses. Three important segments are:

Next: , Previous: Memory Concepts, Up: Memory

3.2 Allocating Storage For Program Data

This section covers how ordinary programs manage storage for their data, including the famous malloc function and some fancier facilities special the GNU C library and GNU Compiler.

Next: , Up: Memory Allocation

3.2.1 Memory Allocation in C Programs

The C language supports two kinds of memory allocation through the variables in C programs:

A third important kind of memory allocation, dynamic allocation, is not supported by C variables but is available via GNU C library functions.

3.2.1.1 Dynamic Memory Allocation

Dynamic memory allocation is a technique in which programs determine as they are running where to store some information. You need dynamic allocation when the amount of memory you need, or how long you continue to need it, depends on factors that are not known before the program runs.

For example, you may need a block to store a line read from an input file; since there is no limit to how long a line can be, you must allocate the memory dynamically and make it dynamically larger as you read more of the line.

Or, you may need a block for each record or each definition in the input data; since you can't know in advance how many there will be, you must allocate a new block for each record or definition as you read it.

When you use dynamic allocation, the allocation of a block of memory is an action that the program requests explicitly. You call a function or macro when you want to allocate space, and specify the size with an argument. If you want to free the space, you do so by calling another function or macro. You can do these things whenever you want, as often as you want.

Dynamic allocation is not supported by C variables; there is no storage class “dynamic”, and there can never be a C variable whose value is stored in dynamically allocated space. The only way to get dynamically allocated memory is via a system call (which is generally via a GNU C library function call), and the only way to refer to dynamically allocated space is through a pointer. Because it is less convenient, and because the actual process of dynamic allocation requires more computation time, programmers generally use dynamic allocation only when neither static nor automatic allocation will serve.

For example, if you want to allocate dynamically some space to hold a struct foobar, you cannot declare a variable of type struct foobar whose contents are the dynamically allocated space. But you can declare a variable of pointer type struct foobar * and assign it the address of the space. Then you can use the operators ‘*’ and ‘->’ on this pointer variable to refer to the contents of the space: 

     {
       struct foobar *ptr
          = (struct foobar *) malloc (sizeof (struct foobar));
       ptr->name = x;
       ptr->next = current_foobar;
       current_foobar = ptr;
     }

Next: , Previous: Memory Allocation and C, Up: Memory Allocation

3.2.2 Unconstrained Allocation

The most general dynamic allocation facility is malloc. It allows you to allocate blocks of memory of any size at any time, make them bigger or smaller at any time, and free the blocks individually at any time (or never).

Next: , Up: Unconstrained Allocation
3.2.2.1 Basic Memory Allocation

To allocate a block of memory, call malloc. The prototype for this function is in stdlib.h.

— Function: void * malloc (size_t size)

This function returns a pointer to a newly allocated block size bytes long, or a null pointer if the block could not be allocated.

The contents of the block are undefined; you must initialize it yourself (or use calloc instead; see Allocating Cleared Space). Normally you would cast the value as a pointer to the kind of object that you want to store in the block. Here we show an example of doing so, and of initializing the space with zeros using the library function memset (see Copying and Concatenation): 

     struct foo *ptr;
     ...
     ptr = (struct foo *) malloc (sizeof (struct foo));
     if (ptr == 0) abort ();
     memset (ptr, 0, sizeof (struct foo));

You can store the result of malloc into any pointer variable without a cast, because ISO C automatically converts the type void * to another type of pointer when necessary. But the cast is necessary in contexts other than assignment operators or if you might want your code to run in traditional C.

Remember that when allocating space for a string, the argument to malloc must be one plus the length of the string. This is because a string is terminated with a null character that doesn't count in the “length” of the string but does need space. For example: 

     char *ptr;
     ...
     ptr = (char *) malloc (length + 1);

See Representation of Strings, for more information about this.

Next: , Previous: Basic Allocation, Up: Unconstrained Allocation
3.2.2.2 Examples of malloc

If no more space is available, malloc returns a null pointer. You should check the value of every call to malloc. It is useful to write a subroutine that calls malloc and reports an error if the value is a null pointer, returning only if the value is nonzero. This function is conventionally called xmalloc. Here it is: 

     void *
     xmalloc (size_t size)
     {
       register void *value = malloc (size);
       if (value == 0)
         fatal ("virtual memory exhausted");
       return value;
     }

Here is a real example of using malloc (by way of xmalloc). The function savestring will copy a sequence of characters into a newly allocated null-terminated string: 

     char *
     savestring (const char *ptr, size_t len)
     {
       register char *value = (char *) xmalloc (len + 1);
       value[len] = '\0';
       return (char *) memcpy (value, ptr, len);
     }

The block that malloc gives you is guaranteed to be aligned so that it can hold any type of data. In the GNU system, the address is always a multiple of eight on most systems, and a multiple of 16 on 64-bit systems. Only rarely is any higher boundary (such as a page boundary) necessary; for those cases, use memalign, posix_memalign or valloc (see Aligned Memory Blocks).

Note that the memory located after the end of the block is likely to be in use for something else; perhaps a block already allocated by another call to malloc. If you attempt to treat the block as longer than you asked for it to be, you are liable to destroy the data that malloc uses to keep track of its blocks, or you may destroy the contents of another block. If you have already allocated a block and discover you want it to be bigger, use realloc (see Changing Block Size).

Next: , Previous: Malloc Examples, Up: Unconstrained Allocation
3.2.2.3 Freeing Memory Allocated with malloc

When you no longer need a block that you got with malloc, use the function free to make the block available to be allocated again. The prototype for this function is in stdlib.h.

— Function: void free (void *ptr)

The free function deallocates the block of memory pointed at by ptr.

— Function: void cfree (void *ptr)

This function does the same thing as free. It's provided for backward compatibility with SunOS; you should use free instead.

Freeing a block alters the contents of the block. Do not expect to find any data (such as a pointer to the next block in a chain of blocks) in the block after freeing it. Copy whatever you need out of the block before freeing it! Here is an example of the proper way to free all the blocks in a chain, and the strings that they point to: 

     struct chain
       {
         struct chain *next;
         char *name;
       }
     
     void
     free_chain (struct chain *chain)
     {
       while (chain != 0)
         {
           struct chain *next = chain->next;
           free (chain->name);
           free (chain);
           chain = next;
         }
     }

Occasionally, free can actually return memory to the operating system and make the process smaller. Usually, all it can do is allow a later call to malloc to reuse the space. In the meantime, the space remains in your program as part of a free-list used internally by malloc.

There is no point in freeing blocks at the end of a program, because all of the program's space is given back to the system when the process terminates.

Next: , Previous: Freeing after Malloc, Up: Unconstrained Allocation
3.2.2.4 Changing the Size of a Block

Often you do not know for certain how big a block you will ultimately need at the time you must begin to use the block. For example, the block might be a buffer that you use to hold a line being read from a file; no matter how long you make the buffer initially, you may encounter a line that is longer.

You can make the block longer by calling realloc. This function is declared in stdlib.h.

— Function: void * realloc (void *ptr, size_t newsize)

The realloc function changes the size of the block whose address is ptr to be newsize.

Since the space after the end of the block may be in use, realloc may find it necessary to copy the block to a new address where more free space is available. The value of realloc is the new address of the block. If the block needs to be moved, realloc copies the old contents.

If you pass a null pointer for ptr, realloc behaves just like ‘malloc (newsize)’. This can be convenient, but beware that older implementations (before ISO C) may not support this behavior, and will probably crash when realloc is passed a null pointer.

Like malloc, realloc may return a null pointer if no memory space is available to make the block bigger. When this happens, the original block is untouched; it has not been modified or relocated.

In most cases it makes no difference what happens to the original block when realloc fails, because the application program cannot continue when it is out of memory, and the only thing to do is to give a fatal error message. Often it is convenient to write and use a subroutine, conventionally called xrealloc, that takes care of the error message as xmalloc does for malloc: 

     void *
     xrealloc (void *ptr, size_t size)
     {
       register void *value = realloc (ptr, size);
       if (value == 0)
         fatal ("Virtual memory exhausted");
       return value;
     }

You can also use realloc to make a block smaller. The reason you would do this is to avoid tying up a lot of memory space when only a little is needed. In several allocation implementations, making a block smaller sometimes necessitates copying it, so it can fail if no other space is available.

If the new size you specify is the same as the old size, realloc is guaranteed to change nothing and return the same address that you gave.

Next: , Previous: Changing Block Size, Up: Unconstrained Allocation
3.2.2.5 Allocating Cleared Space

The function calloc allocates memory and clears it to zero. It is declared in stdlib.h.

— Function: void * calloc (size_t count, size_t eltsize)

This function allocates a block long enough to contain a vector of count elements, each of size eltsize. Its contents are cleared to zero before calloc returns.

You could define calloc as follows: 

     void *
     calloc (size_t count, size_t eltsize)
     {
       size_t size = count * eltsize;
       void *value = malloc (size);
       if (value != 0)
         memset (value, 0, size);
       return value;
     }

But in general, it is not guaranteed that calloc calls malloc internally. Therefore, if an application provides its own malloc/realloc/free outside the C library, it should always define calloc, too.

Next: , Previous: Allocating Cleared Space, Up: Unconstrained Allocation
3.2.2.6 Efficiency Considerations for malloc

As opposed to other versions, the malloc in the GNU C Library does not round up block sizes to powers of two, neither for large nor for small sizes. Neighboring chunks can be coalesced on a free no matter what their size is. This makes the implementation suitable for all kinds of allocation patterns without generally incurring high memory waste through fragmentation.

Very large blocks (much larger than a page) are allocated with mmap (anonymous or via /dev/zero) by this implementation. This has the great advantage that these chunks are returned to the system immediately when they are freed. Therefore, it cannot happen that a large chunk becomes “locked” in between smaller ones and even after calling free wastes memory. The size threshold for mmap to be used can be adjusted with mallopt. The use of mmap can also be disabled completely.

Next: , Previous: Efficiency and Malloc, Up: Unconstrained Allocation
3.2.2.7 Allocating Aligned Memory Blocks

The address of a block returned by malloc or realloc in the GNU system is always a multiple of eight (or sixteen on 64-bit systems). If you need a block whose address is a multiple of a higher power of two than that, use memalign, posix_memalign, or valloc. memalign is declared in malloc.h and posix_memalign is declared in stdlib.h.

With the GNU library, you can use free to free the blocks that memalign, posix_memalign, and valloc return. That does not work in BSD, however—BSD does not provide any way to free such blocks.

— Function: void * memalign (size_t boundary, size_t size)

The memalign function allocates a block of size bytes whose address is a multiple of boundary. The boundary must be a power of two! The function memalign works by allocating a somewhat larger block, and then returning an address within the block that is on the specified boundary.

— Function: int posix_memalign (void **memptr, size_t alignment, size_t size)

The posix_memalign function is similar to the memalign function in that it returns a buffer of size bytes aligned to a multiple of alignment. But it adds one requirement to the parameter alignment: the value must be a power of two multiple of sizeof (void *).

If the function succeeds in allocation memory a pointer to the allocated memory is returned in *memptr and the return value is zero. Otherwise the function returns an error value indicating the problem.

This function was introduced in POSIX 1003.1d.

— Function: void * valloc (size_t size)

Using valloc is like using memalign and passing the page size as the value of the second argument. It is implemented like this: 

          void *
          valloc (size_t size)
          {
            return memalign (getpagesize (), size);
          }

Query Memory Parameters for more information about the memory subsystem.

Next: , Previous: Aligned Memory Blocks, Up: Unconstrained Allocation
3.2.2.8 Malloc Tunable Parameters

You can adjust some parameters for dynamic memory allocation with the mallopt function. This function is the general SVID/XPG interface, defined in malloc.h.

— Function: int mallopt (int param, int value)

When calling mallopt, the param argument specifies the parameter to be set, and value the new value to be set. Possible choices for param, as defined in malloc.h, are:

M_TRIM_THRESHOLD
This is the minimum size (in bytes) of the top-most, releasable chunk that will cause sbrk to be called with a negative argument in order to return memory to the system.
M_TOP_PAD
This parameter determines the amount of extra memory to obtain from the system when a call to sbrk is required. It also specifies the number of bytes to retain when shrinking the heap by calling sbrk with a negative argument. This provides the necessary hysteresis in heap size such that excessive amounts of system calls can be avoided.
M_MMAP_THRESHOLD
All chunks larger than this value are allocated outside the normal heap, using the mmap system call. This way it is guaranteed that the memory for these chunks can be returned to the system on free. Note that requests smaller than this threshold might still be allocated via mmap.
M_MMAP_MAX
The maximum number of chunks to allocate with mmap. Setting this to zero disables all use of mmap.

Next: , Previous: Malloc Tunable Parameters, Up: Unconstrained Allocation
3.2.2.9 Heap Consistency Checking

You can ask malloc to check the consistency of dynamic memory by using the mcheck function. This function is a GNU extension, declared in mcheck.h.

— Function: int mcheck (void (*abortfn) (enum mcheck_status status))

Calling mcheck tells malloc to perform occasional consistency checks. These will catch things such as writing past the end of a block that was allocated with malloc.

The abortfn argument is the function to call when an inconsistency is found. If you supply a null pointer, then mcheck uses a default function which prints a message and calls abort (see Aborting a Program). The function you supply is called with one argument, which says what sort of inconsistency was detected; its type is described below.

It is too late to begin allocation checking once you have allocated anything with malloc. So mcheck does nothing in that case. The function returns -1 if you call it too late, and 0 otherwise (when it is successful).

The easiest way to arrange to call mcheck early enough is to use the option ‘-lmcheck’ when you link your program; then you don't need to modify your program source at all. Alternatively you might use a debugger to insert a call to mcheck whenever the program is started, for example these gdb commands will automatically call mcheck whenever the program starts: 

          (gdb) break main
          Breakpoint 1, main (argc=2, argv=0xbffff964) at whatever.c:10
          (gdb) command 1
          Type commands for when breakpoint 1 is hit, one per line.
          End with a line saying just "end".
          >call mcheck(0)
          >continue
          >end
          (gdb) ...

This will however only work if no initialization function of any object involved calls any of the malloc functions since mcheck must be called before the first such function.

— Function: enum mcheck_status mprobe (void *pointer)

The mprobe function lets you explicitly check for inconsistencies in a particular allocated block. You must have already called mcheck at the beginning of the program, to do its occasional checks; calling mprobe requests an additional consistency check to be done at the time of the call.

The argument pointer must be a pointer returned by malloc or realloc. mprobe returns a value that says what inconsistency, if any, was found. The values are described below.

— Data Type: enum mcheck_status

This enumerated type describes what kind of inconsistency was detected in an allocated block, if any. Here are the possible values:

MCHECK_DISABLED
mcheck was not called before the first allocation. No consistency checking can be done.
MCHECK_OK
No inconsistency detected.
MCHECK_HEAD
The data immediately before the block was modified. This commonly happens when an array index or pointer is decremented too far.
MCHECK_TAIL
The data immediately after the block was modified. This commonly happens when an array index or pointer is incremented too far.
MCHECK_FREE
The block was already freed.

Another possibility to check for and guard against bugs in the use of malloc, realloc and free is to set the environment variable MALLOC_CHECK_. When MALLOC_CHECK_ is set, a special (less efficient) implementation is used which is designed to be tolerant against simple errors, such as double calls of free with the same argument, or overruns of a single byte (off-by-one bugs). Not all such errors can be protected against, however, and memory leaks can result. If MALLOC_CHECK_ is set to 0, any detected heap corruption is silently ignored; if set to 1, a diagnostic is printed on stderr; if set to 2, abort is called immediately. This can be useful because otherwise a crash may happen much later, and the true cause for the problem is then very hard to track down.

There is one problem with MALLOC_CHECK_: in SUID or SGID binaries it could possibly be exploited since diverging from the normal programs behavior it now writes something to the standard error descriptor. Therefore the use of MALLOC_CHECK_ is disabled by default for SUID and SGID binaries. It can be enabled again by the system administrator by adding a file /etc/suid-debug (the content is not important it could be empty).

So, what's the difference between using MALLOC_CHECK_ and linking with ‘-lmcheck’? MALLOC_CHECK_ is orthogonal with respect to ‘-lmcheck’. ‘-lmcheck’ has been added for backward compatibility. Both MALLOC_CHECK_ and ‘-lmcheck’ should uncover the same bugs - but using MALLOC_CHECK_ you don't need to recompile your application.

Next: , Previous: Heap Consistency Checking, Up: Unconstrained Allocation
3.2.2.10 Memory Allocation Hooks

The GNU C library lets you modify the behavior of malloc, realloc, and free by specifying appropriate hook functions. You can use these hooks to help you debug programs that use dynamic memory allocation, for example.

The hook variables are declared in malloc.h.

— Variable: __malloc_hook

The value of this variable is a pointer to the function that malloc uses whenever it is called. You should define this function to look like malloc; that is, like: 

          void *function (size_t size, const void *caller)

The value of caller is the return address found on the stack when the malloc function was called. This value allows you to trace the memory consumption of the program.

— Variable: __realloc_hook

The value of this variable is a pointer to function that realloc uses whenever it is called. You should define this function to look like realloc; that is, like: 

          void *function (void *ptr, size_t size, const void *caller)

The value of caller is the return address found on the stack when the realloc function was called. This value allows you to trace the memory consumption of the program.

— Variable: __free_hook

The value of this variable is a pointer to function that free uses whenever it is called. You should define this function to look like free; that is, like: 

          void function (void *ptr, const void *caller)

The value of caller is the return address found on the stack when the free function was called. This value allows you to trace the memory consumption of the program.

— Variable: __memalign_hook

The value of this variable is a pointer to function that memalign uses whenever it is called. You should define this function to look like memalign; that is, like: 

          void *function (size_t alignment, size_t size, const void *caller)

The value of caller is the return address found on the stack when the memalign function was called. This value allows you to trace the memory consumption of the program.

You must make sure that the function you install as a hook for one of these functions does not call that function recursively without restoring the old value of the hook first! Otherwise, your program will get stuck in an infinite recursion. Before calling the function recursively, one should make sure to restore all the hooks to their previous value. When coming back from the recursive call, all the hooks should be resaved since a hook might modify itself.

— Variable: __malloc_initialize_hook

The value of this variable is a pointer to a function that is called once when the malloc implementation is initialized. This is a weak variable, so it can be overridden in the application with a definition like the following: 

          void (*__malloc_initialize_hook) (void) = my_init_hook;

An issue to look out for is the time at which the malloc hook functions can be safely installed. If the hook functions call the malloc-related functions recursively, it is necessary that malloc has already properly initialized itself at the time when __malloc_hook etc. is assigned to. On the other hand, if the hook functions provide a complete malloc implementation of their own, it is vital that the hooks are assigned to before the very first malloc call has completed, because otherwise a chunk obtained from the ordinary, un-hooked malloc may later be handed to __free_hook, for example.

In both cases, the problem can be solved by setting up the hooks from within a user-defined function pointed to by __malloc_initialize_hook—then the hooks will be set up safely at the right time.

Here is an example showing how to use __malloc_hook and __free_hook properly. It installs a function that prints out information every time malloc or free is called. We just assume here that realloc and memalign are not used in our program. 

     /* Prototypes for __malloc_hook, __free_hook */
     #include <malloc.h>
     
     /* Prototypes for our hooks.  */
     static void my_init_hook (void);
     static void *my_malloc_hook (size_t, const void *);
     static void my_free_hook (void*, const void *);
     
     /* Override initializing hook from the C library. */
     void (*__malloc_initialize_hook) (void) = my_init_hook;
     
     static void
     my_init_hook (void)
     {
       old_malloc_hook = __malloc_hook;
       old_free_hook = __free_hook;
       __malloc_hook = my_malloc_hook;
       __free_hook = my_free_hook;
     }
     
     static void *
     my_malloc_hook (size_t size, const void *caller)
     {
       void *result;
       /* Restore all old hooks */
       __malloc_hook = old_malloc_hook;
       __free_hook = old_free_hook;
       /* Call recursively */
       result = malloc (size);
       /* Save underlying hooks */
       old_malloc_hook = __malloc_hook;
       old_free_hook = __free_hook;
       /* printf might call malloc, so protect it too. */
       printf ("malloc (%u) returns %p\n", (unsigned int) size, result);
       /* Restore our own hooks */
       __malloc_hook = my_malloc_hook;
       __free_hook = my_free_hook;
       return result;
     }
     
     static void
     my_free_hook (void *ptr, const void *caller)
     {
       /* Restore all old hooks */
       __malloc_hook = old_malloc_hook;
       __free_hook = old_free_hook;
       /* Call recursively */
       free (ptr);
       /* Save underlying hooks */
       old_malloc_hook = __malloc_hook;
       old_free_hook = __free_hook;
       /* printf might call free, so protect it too. */
       printf ("freed pointer %p\n", ptr);
       /* Restore our own hooks */
       __malloc_hook = my_malloc_hook;
       __free_hook = my_free_hook;
     }
     
     main ()
     {
       ...
     }

The mcheck function (see Heap Consistency Checking) works by installing such hooks.

Next: , Previous: Hooks for Malloc, Up: Unconstrained Allocation
3.2.2.11 Statistics for Memory Allocation with malloc

You can get information about dynamic memory allocation by calling the mallinfo function. This function and its associated data type are declared in malloc.h; they are an extension of the standard SVID/XPG version.

— Data Type: struct mallinfo

This structure type is used to return information about the dynamic memory allocator. It contains the following members:

int arena
This is the total size of memory allocated with sbrk by malloc, in bytes.
int ordblks
This is the number of chunks not in use. (The memory allocator internally gets chunks of memory from the operating system, and then carves them up to satisfy individual malloc requests; see Efficiency and Malloc.)
int smblks
This field is unused.
int hblks
This is the total number of chunks allocated with mmap.
int hblkhd
This is the total size of memory allocated with mmap, in bytes.
int usmblks
This field is unused.
int fsmblks
This field is unused.
int uordblks
This is the total size of memory occupied by chunks handed out by malloc.
int fordblks
This is the total size of memory occupied by free (not in use) chunks.
int keepcost
This is the size of the top-most releasable chunk that normally borders the end of the heap (i.e., the high end of the virtual address space's data segment).

— Function: struct mallinfo mallinfo (void)

This function returns information about the current dynamic memory usage in a structure of type struct mallinfo.

Previous: Statistics of Malloc, Up: Unconstrained Allocation
3.2.2.12 Summary of malloc-Related Functions

Here is a summary of the functions that work with malloc:

void *malloc (size_t size)
Allocate a block of size bytes. See Basic Allocation.
void free (void *addr)
Free a block previously allocated by malloc. See Freeing after Malloc.
void *realloc (void *addr, size_t size)
Make a block previously allocated by malloc larger or smaller, possibly by copying it to a new location. See Changing Block Size.
void *calloc (size_t count, size_t eltsize)
Allocate a block of count * eltsize bytes using malloc, and set its contents to zero. See Allocating Cleared Space.
void *valloc (size_t size)
Allocate a block of size bytes, starting on a page boundary. See Aligned Memory Blocks.
void *memalign (size_t size, size_t boundary)
Allocate a block of size bytes, starting on an address that is a multiple of boundary. See Aligned Memory Blocks.
int mallopt (int param, int value)
Adjust a tunable parameter. See Malloc Tunable Parameters.
int mcheck (void (*abortfn) (void))
Tell malloc to perform occasional consistency checks on dynamically allocated memory, and to call abortfn when an inconsistency is found. See Heap Consistency Checking.
void *(*__malloc_hook) (size_t size, const void *caller)
A pointer to a function that malloc uses whenever it is called.
void *(*__realloc_hook) (void *ptr, size_t size, const void *caller)
A pointer to a function that realloc uses whenever it is called.
void (*__free_hook) (void *ptr, const void *caller)
A pointer to a function that free uses whenever it is called.
void (*__memalign_hook) (size_t size, size_t alignment, const void *caller)
A pointer to a function that memalign uses whenever it is called.
struct mallinfo mallinfo (void)
Return information about the current dynamic memory usage. See Statistics of Malloc.

Next: , Previous: Unconstrained Allocation, Up: Memory Allocation

3.2.3 Allocation Debugging

A complicated task when programming with languages which do not use garbage collected dynamic memory allocation is to find memory leaks. Long running programs must assure that dynamically allocated objects are freed at the end of their lifetime. If this does not happen the system runs out of memory, sooner or later.

The malloc implementation in the GNU C library provides some simple means to detect such leaks and obtain some information to find the location. To do this the application must be started in a special mode which is enabled by an environment variable. There are no speed penalties for the program if the debugging mode is not enabled.

Next: , Up: Allocation Debugging
3.2.3.1 How to install the tracing functionality
— Function: void mtrace (void)

When the mtrace function is called it looks for an environment variable named MALLOC_TRACE. This variable is supposed to contain a valid file name. The user must have write access. If the file already exists it is truncated. If the environment variable is not set or it does not name a valid file which can be opened for writing nothing is done. The behavior of malloc etc. is not changed. For obvious reasons this also happens if the application is installed with the SUID or SGID bit set.

If the named file is successfully opened, mtrace installs special handlers for the functions malloc, realloc, and free (see Hooks for Malloc). From then on, all uses of these functions are traced and protocolled into the file. There is now of course a speed penalty for all calls to the traced functions so tracing should not be enabled during normal use.

This function is a GNU extension and generally not available on other systems. The prototype can be found in mcheck.h.

— Function: void muntrace (void)

The muntrace function can be called after mtrace was used to enable tracing the malloc calls. If no (successful) call of mtrace was made muntrace does nothing.

Otherwise it deinstalls the handlers for malloc, realloc, and free and then closes the protocol file. No calls are protocolled anymore and the program runs again at full speed.

This function is a GNU extension and generally not available on other systems. The prototype can be found in mcheck.h.

Next: , Previous: Tracing malloc, Up: Allocation Debugging
3.2.3.2 Example program excerpts

Even though the tracing functionality does not influence the runtime behavior of the program it is not a good idea to call mtrace in all programs. Just imagine that you debug a program using mtrace and all other programs used in the debugging session also trace their malloc calls. The output file would be the same for all programs and thus is unusable. Therefore one should call mtrace only if compiled for debugging. A program could therefore start like this: 

     #include <mcheck.h>
     
     int
     main (int argc, char *argv[])
     {
     #ifdef DEBUGGING
       mtrace ();
     #endif
       ...
     }

This is all what is needed if you want to trace the calls during the whole runtime of the program. Alternatively you can stop the tracing at any time with a call to muntrace. It is even possible to restart the tracing again with a new call to mtrace. But this can cause unreliable results since there may be calls of the functions which are not called. Please note that not only the application uses the traced functions, also libraries (including the C library itself) use these functions.

This last point is also why it is no good idea to call muntrace before the program terminated. The libraries are informed about the termination of the program only after the program returns from main or calls exit and so cannot free the memory they use before this time.

So the best thing one can do is to call mtrace as the very first function in the program and never call muntrace. So the program traces almost all uses of the malloc functions (except those calls which are executed by constructors of the program or used libraries).

Next: , Previous: Using the Memory Debugger, Up: Allocation Debugging
3.2.3.3 Some more or less clever ideas

You know the situation. The program is prepared for debugging and in all debugging sessions it runs well. But once it is started without debugging the error shows up. A typical example is a memory leak that becomes visible only when we turn off the debugging. If you foresee such situations you can still win. Simply use something equivalent to the following little program: 

     #include <mcheck.h>
     #include <signal.h>
     
     static void
     enable (int sig)
     {
       mtrace ();
       signal (SIGUSR1, enable);
     }
     
     static void
     disable (int sig)
     {
       muntrace ();
       signal (SIGUSR2, disable);
     }
     
     int
     main (int argc, char *argv[])
     {
       ...
     
       signal (SIGUSR1, enable);
       signal (SIGUSR2, disable);
     
       ...
     }

I.e., the user can start the memory debugger any time s/he wants if the program was started with MALLOC_TRACE set in the environment. The output will of course not show the allocations which happened before the first signal but if there is a memory leak this will show up nevertheless.

Previous: Tips for the Memory Debugger, Up: Allocation Debugging
3.2.3.4 Interpreting the traces

If you take a look at the output it will look similar to this: 

     = Start
      [0x8048209] - 0x8064cc8
      [0x8048209] - 0x8064ce0
      [0x8048209] - 0x8064cf8
      [0x80481eb] + 0x8064c48 0x14
      [0x80481eb] + 0x8064c60 0x14
      [0x80481eb] + 0x8064c78 0x14
      [0x80481eb] + 0x8064c90 0x14
     = End

What this all means is not really important since the trace file is not meant to be read by a human. Therefore no attention is given to readability. Instead there is a program which comes with the GNU C library which interprets the traces and outputs a summary in an user-friendly way. The program is called mtrace (it is in fact a Perl script) and it takes one or two arguments. In any case the name of the file with the trace output must be specified. If an optional argument precedes the name of the trace file this must be the name of the program which generated the trace. 

     drepper$ mtrace tst-mtrace log
     No memory leaks.

In this case the program tst-mtrace was run and it produced a trace file log. The message printed by mtrace shows there are no problems with the code, all allocated memory was freed afterwards.

If we call mtrace on the example trace given above we would get a different outout: 

     drepper$ mtrace errlog
     - 0x08064cc8 Free 2 was never alloc'd 0x8048209
     - 0x08064ce0 Free 3 was never alloc'd 0x8048209
     - 0x08064cf8 Free 4 was never alloc'd 0x8048209
     
     Memory not freed:
     -----------------
        Address     Size     Caller
     0x08064c48     0x14  at 0x80481eb
     0x08064c60     0x14  at 0x80481eb
     0x08064c78     0x14  at 0x80481eb
     0x08064c90     0x14  at 0x80481eb

We have called mtrace with only one argument and so the script has no chance to find out what is meant with the addresses given in the trace. We can do better: 

     drepper$ mtrace tst errlog
     - 0x08064cc8 Free 2 was never alloc'd /home/drepper/tst.c:39
     - 0x08064ce0 Free 3 was never alloc'd /home/drepper/tst.c:39
     - 0x08064cf8 Free 4 was never alloc'd /home/drepper/tst.c:39
     
     Memory not freed:
     -----------------
        Address     Size     Caller
     0x08064c48     0x14  at /home/drepper/tst.c:33
     0x08064c60     0x14  at /home/drepper/tst.c:33
     0x08064c78     0x14  at /home/drepper/tst.c:33
     0x08064c90     0x14  at /home/drepper/tst.c:33

Suddenly the output makes much more sense and the user can see immediately where the function calls causing the trouble can be found.

Interpreting this output is not complicated. There are at most two different situations being detected. First, free was called for pointers which were never returned by one of the allocation functions. This is usually a very bad problem and what this looks like is shown in the first three lines of the output. Situations like this are quite rare and if they appear they show up very drastically: the program normally crashes.

The other situation which is much harder to detect are memory leaks. As you can see in the output the mtrace function collects all this information and so can say that the program calls an allocation function from line 33 in the source file /home/drepper/tst-mtrace.c four times without freeing this memory before the program terminates. Whether this is a real problem remains to be investigated.

Next: , Previous: Allocation Debugging, Up: Memory Allocation

3.2.4 Obstacks

An obstack is a pool of memory containing a stack of objects. You can create any number of separate obstacks, and then allocate objects in specified obstacks. Within each obstack, the last object allocated must always be the first one freed, but distinct obstacks are independent of each other.

Aside from this one constraint of order of freeing, obstacks are totally general: an obstack can contain any number of objects of any size. They are implemented with macros, so allocation is usually very fast as long as the objects are usually small. And the only space overhead per object is the padding needed to start each object on a suitable boundary.

Next: , Up: Obstacks
3.2.4.1 Creating Obstacks

The utilities for manipulating obstacks are declared in the header file obstack.h.

— Data Type: struct obstack

An obstack is represented by a data structure of type struct obstack. This structure has a small fixed size; it records the status of the obstack and how to find the space in which objects are allocated. It does not contain any of the objects themselves. You should not try to access the contents of the structure directly; use only the functions described in this chapter.

You can declare variables of type struct obstack and use them as obstacks, or you can allocate obstacks dynamically like any other kind of object. Dynamic allocation of obstacks allows your program to have a variable number of different stacks. (You can even allocate an obstack structure in another obstack, but this is rarely useful.)

All the functions that work with obstacks require you to specify which obstack to use. You do this with a pointer of type struct obstack *. In the following, we often say “an obstack” when strictly speaking the object at hand is such a pointer.

The objects in the obstack are packed into large blocks called chunks. The struct obstack structure points to a chain of the chunks currently in use.

The obstack library obtains a new chunk whenever you allocate an object that won't fit in the previous chunk. Since the obstack library manages chunks automatically, you don't need to pay much attention to them, but you do need to supply a function which the obstack library should use to get a chunk. Usually you supply a function which uses malloc directly or indirectly. You must also supply a function to free a chunk. These matters are described in the following section.

Next: , Previous: Creating Obstacks, Up: Obstacks
3.2.4.2 Preparing for Using Obstacks

Each source file in which you plan to use the obstack functions must include the header file obstack.h, like this: 

     #include <obstack.h>

Also, if the source file uses the macro obstack_init, it must declare or define two functions or macros that will be called by the obstack library. One, obstack_chunk_alloc, is used to allocate the chunks of memory into which objects are packed. The other, obstack_chunk_free, is used to return chunks when the objects in them are freed. These macros should appear before any use of obstacks in the source file.

Usually these are defined to use malloc via the intermediary xmalloc (see Unconstrained Allocation). This is done with the following pair of macro definitions: 

     #define obstack_chunk_alloc xmalloc
     #define obstack_chunk_free free

Though the memory you get using obstacks really comes from malloc, using obstacks is faster because malloc is called less often, for larger blocks of memory. See Obstack Chunks, for full details.

At run time, before the program can use a struct obstack object as an obstack, it must initialize the obstack by calling obstack_init.

— Function: int obstack_init (struct obstack *obstack-ptr)

Initialize obstack obstack-ptr for allocation of objects. This function calls the obstack's obstack_chunk_alloc function. If allocation of memory fails, the function pointed to by obstack_alloc_failed_handler is called. The obstack_init function always returns 1 (Compatibility notice: Former versions of obstack returned 0 if allocation failed).

Here are two examples of how to allocate the space for an obstack and initialize it. First, an obstack that is a static variable: 

     static struct obstack myobstack;
     ...
     obstack_init (&myobstack);

Second, an obstack that is itself dynamically allocated: 

     struct obstack *myobstack_ptr
       = (struct obstack *) xmalloc (sizeof (struct obstack));
     
     obstack_init (myobstack_ptr);
— Variable: obstack_alloc_failed_handler

The value of this variable is a pointer to a function that obstack uses when obstack_chunk_alloc fails to allocate memory. The default action is to print a message and abort. You should supply a function that either calls exit (see Program Termination) or longjmp (see Non-Local Exits) and doesn't return. 

          void my_obstack_alloc_failed (void)
          ...
          obstack_alloc_failed_handler = &my_obstack_alloc_failed;

Next: , Previous: Preparing for Obstacks, Up: Obstacks
3.2.4.3 Allocation in an Obstack

The most direct way to allocate an object in an obstack is with obstack_alloc, which is invoked almost like malloc.

— Function: void * obstack_alloc (struct obstack *obstack-ptr, int size)

This allocates an uninitialized block of size bytes in an obstack and returns its address. Here obstack-ptr specifies which obstack to allocate the block in; it is the address of the struct obstack object which represents the obstack. Each obstack function or macro requires you to specify an obstack-ptr as the first argument.

This function calls the obstack's obstack_chunk_alloc function if it needs to allocate a new chunk of memory; it calls obstack_alloc_failed_handler if allocation of memory by obstack_chunk_alloc failed.

For example, here is a function that allocates a copy of a string str in a specific obstack, which is in the variable string_obstack: 

     struct obstack string_obstack;
     
     char *
     copystring (char *string)
     {
       size_t len = strlen (string) + 1;
       char *s = (char *) obstack_alloc (&string_obstack, len);
       memcpy (s, string, len);
       return s;
     }

To allocate a block with specified contents, use the function obstack_copy, declared like this:

— Function: void * obstack_copy (struct obstack *obstack-ptr, void *address, int size)

This allocates a block and initializes it by copying size bytes of data starting at address. It calls obstack_alloc_failed_handler if allocation of memory by obstack_chunk_alloc failed.

— Function: void * obstack_copy0 (struct obstack *obstack-ptr, void *address, int size)

Like obstack_copy, but appends an extra byte containing a null character. This extra byte is not counted in the argument size.

The obstack_copy0 function is convenient for copying a sequence of characters into an obstack as a null-terminated string. Here is an example of its use: 

     char *
     obstack_savestring (char *addr, int size)
     {
       return obstack_copy0 (&myobstack, addr, size);
     }

Contrast this with the previous example of savestring using malloc (see Basic Allocation).

Next: , Previous: Allocation in an Obstack, Up: Obstacks
3.2.4.4 Freeing Objects in an Obstack

To free an object allocated in an obstack, use the function obstack_free. Since the obstack is a stack of objects, freeing one object automatically frees all other objects allocated more recently in the same obstack.

— Function: void obstack_free (struct obstack *obstack-ptr, void *object)

If object is a null pointer, everything allocated in the obstack is freed. Otherwise, object must be the address of an object allocated in the obstack. Then object is freed, along with everything allocated in obstack since object.

Note that if object is a null pointer, the result is an uninitialized obstack. To free all memory in an obstack but leave it valid for further allocation, call obstack_free with the address of the first object allocated on the obstack: 

     obstack_free (obstack_ptr, first_object_allocated_ptr);

Recall that the objects in an obstack are grouped into chunks. When all the objects in a chunk become free, the obstack library automatically frees the chunk (see Preparing for Obstacks). Then other obstacks, or non-obstack allocation, can reuse the space of the chunk.

Next: , Previous: Freeing Obstack Objects, Up: Obstacks
3.2.4.5 Obstack Functions and Macros

The interfaces for using obstacks may be defined either as functions or as macros, depending on the compiler. The obstack facility works with all C compilers, including both ISO C and traditional C, but there are precautions you must take if you plan to use compilers other than GNU C.

If you are using an old-fashioned non-ISO C compiler, all the obstack “functions” are actually defined only as macros. You can call these macros like functions, but you cannot use them in any other way (for example, you cannot take their address).

Calling the macros requires a special precaution: namely, the first operand (the obstack pointer) may not contain any side effects, because it may be computed more than once. For example, if you write this: 

     obstack_alloc (get_obstack (), 4);

you will find that get_obstack may be called several times. If you use *obstack_list_ptr++ as the obstack pointer argument, you will get very strange results since the incrementation may occur several times.

In ISO C, each function has both a macro definition and a function definition. The function definition is used if you take the address of the function without calling it. An ordinary call uses the macro definition by default, but you can request the function definition instead by writing the function name in parentheses, as shown here: 

     char *x;
     void *(*funcp) ();
     /* Use the macro.  */
     x = (char *) obstack_alloc (obptr, size);
     /* Call the function.  */
     x = (char *) (obstack_alloc) (obptr, size);
     /* Take the address of the function.  */
     funcp = obstack_alloc;

This is the same situation that exists in ISO C for the standard library functions. See Macro Definitions.

Warning: When you do use the macros, you must observe the precaution of avoiding side effects in the first operand, even in ISO C.

If you use the GNU C compiler, this precaution is not necessary, because various language extensions in GNU C permit defining the macros so as to compute each argument only once.

Next: , Previous: Obstack Functions, Up: Obstacks
3.2.4.6 Growing Objects

Because memory in obstack chunks is used sequentially, it is possible to build up an object step by step, adding one or more bytes at a time to the end of the object. With this technique, you do not need to know how much data you will put in the object until you come to the end of it. We call this the technique of growing objects. The special functions for adding data to the growing object are described in this section.

You don't need to do anything special when you start to grow an object. Using one of the functions to add data to the object automatically starts it. However, it is necessary to say explicitly when the object is finished. This is done with the function obstack_finish.

The actual address of the object thus built up is not known until the object is finished. Until then, it always remains possible that you will add so much data that the object must be copied into a new chunk.

While the obstack is in use for a growing object, you cannot use it for ordinary allocation of another object. If you try to do so, the space already added to the growing object will become part of the other object.

— Function: void obstack_blank (struct obstack *obstack-ptr, int size)

The most basic function for adding to a growing object is obstack_blank, which adds space without initializing it.

— Function: void obstack_grow (struct obstack *obstack-ptr, void *data, int size)

To add a block of initialized space, use obstack_grow, which is the growing-object analogue of obstack_copy. It adds size bytes of data to the growing object, copying the contents from data.

— Function: void obstack_grow0 (struct obstack *obstack-ptr, void *data, int size)

This is the growing-object analogue of obstack_copy0. It adds size bytes copied from data, followed by an additional null character.

— Function: void obstack_1grow (struct obstack *obstack-ptr, char c)

To add one character at a time, use the function obstack_1grow. It adds a single byte containing c to the growing object.

— Function: void obstack_ptr_grow (struct obstack *obstack-ptr, void *data)

Adding the value of a pointer one can use the function obstack_ptr_grow. It adds sizeof (void *) bytes containing the value of data.

— Function: void obstack_int_grow (struct obstack *obstack-ptr, int data)

A single value of type int can be added by using the obstack_int_grow function. It adds sizeof (int) bytes to the growing object and initializes them with the value of data.

— Function: void * obstack_finish (struct obstack *obstack-ptr)

When you are finished growing the object, use the function obstack_finish to close it off and return its final address.

Once you have finished the object, the obstack is available for ordinary allocation or for growing another object.

This function can return a null pointer under the same conditions as obstack_alloc (see Allocation in an Obstack).

When you build an object by growing it, you will probably need to know afterward how long it became. You need not keep track of this as you grow the object, because you can find out the length from the obstack just before finishing the object with the function obstack_object_size, declared as follows:

— Function: int obstack_object_size (struct obstack *obstack-ptr)

This function returns the current size of the growing object, in bytes. Remember to call this function before finishing the object. After it is finished, obstack_object_size will return zero.

If you have started growing an object and wish to cancel it, you should finish it and then free it, like this: 

     obstack_free (obstack_ptr, obstack_finish (obstack_ptr));

This has no effect if no object was growing.

You can use obstack_blank with a negative size argument to make the current object smaller. Just don't try to shrink it beyond zero length—there's no telling what will happen if you do that.

Next: , Previous: Growing Objects, Up: Obstacks
3.2.4.7 Extra Fast Growing Objects

The usual functions for growing objects incur overhead for checking whether there is room for the new growth in the current chunk. If you are frequently constructing objects in small steps of growth, this overhead can be significant.

You can reduce the overhead by using special “fast growth” functions that grow the object without checking. In order to have a robust program, you must do the checking yourself. If you do this checking in the simplest way each time you are about to add data to the object, you have not saved anything, because that is what the ordinary growth functions do. But if you can arrange to check less often, or check more efficiently, then you make the program faster.

The function obstack_room returns the amount of room available in the current chunk. It is declared as follows:

— Function: int obstack_room (struct obstack *obstack-ptr)

This returns the number of bytes that can be added safely to the current growing object (or to an object about to be started) in obstack obstack using the fast growth functions.

While you know there is room, you can use these fast growth functions for adding data to a growing object:

— Function: void obstack_1grow_fast (struct obstack *obstack-ptr, char c)

The function obstack_1grow_fast adds one byte containing the character c to the growing object in obstack obstack-ptr.

— Function: void obstack_ptr_grow_fast (struct obstack *obstack-ptr, void *data)

The function obstack_ptr_grow_fast adds sizeof (void *) bytes containing the value of data to the growing object in obstack obstack-ptr.

— Function: void obstack_int_grow_fast (struct obstack *obstack-ptr, int data)

The function obstack_int_grow_fast adds sizeof (int) bytes containing the value of data to the growing object in obstack obstack-ptr.

— Function: void obstack_blank_fast (struct obstack *obstack-ptr, int size)

The function obstack_blank_fast adds size bytes to the growing object in obstack obstack-ptr without initializing them.

When you check for space using obstack_room and there is not enough room for what you want to add, the fast growth functions are not safe. In this case, simply use the corresponding ordinary growth function instead. Very soon this will copy the object to a new chunk; then there will be lots of room available again.

So, each time you use an ordinary growth function, check afterward for sufficient space using obstack_room. Once the object is copied to a new chunk, there will be plenty of space again, so the program will start using the fast growth functions again.

Here is an example: 

     void
     add_string (struct obstack *obstack, const char *ptr, int len)
     {
       while (len > 0)
         {
           int room = obstack_room (obstack);
           if (room == 0)
             {
               /* Not enough room. Add one character slowly,
                  which may copy to a new chunk and make room.  */
               obstack_1grow (obstack, *ptr++);
               len--;
             }
           else
             {
               if (room > len)
                 room = len;
               /* Add fast as much as we have room for. */
               len -= room;
               while (room-- > 0)
                 obstack_1grow_fast (obstack, *ptr++);
             }
         }
     }

Next: , Previous: Extra Fast Growing, Up: Obstacks
3.2.4.8 Status of an Obstack

Here are functions that provide information on the current status of allocation in an obstack. You can use them to learn about an object while still growing it.

— Function: void * obstack_base (struct obstack *obstack-ptr)

This function returns the tentative address of the beginning of the currently growing object in obstack-ptr. If you finish the object immediately, it will have that address. If you make it larger first, it may outgrow the current chunk—then its address will change!

If no object is growing, this value says where the next object you allocate will start (once again assuming it fits in the current chunk).

— Function: void * obstack_next_free (struct obstack *obstack-ptr)

This function returns the address of the first free byte in the current chunk of obstack obstack-ptr. This is the end of the currently growing object. If no object is growing, obstack_next_free returns the same value as obstack_base.

— Function: int obstack_object_size (struct obstack *obstack-ptr)

This function returns the size in bytes of the currently growing object. This is equivalent to 

          obstack_next_free (obstack-ptr) - obstack_base (obstack-ptr)

Next: , Previous: Status of an Obstack, Up: Obstacks
3.2.4.9 Alignment of Data in Obstacks

Each obstack has an alignment boundary; each object allocated in the obstack automatically starts on an address that is a multiple of the specified boundary. By default, this boundary is aligned so that the object can hold any type of data.

To access an obstack's alignment boundary, use the macro obstack_alignment_mask, whose function prototype looks like this:

— Macro: int obstack_alignment_mask (struct obstack *obstack-ptr)

The value is a bit mask; a bit that is 1 indicates that the corresponding bit in the address of an object should be 0. The mask value should be one less than a power of 2; the effect is that all object addresses are multiples of that power of 2. The default value of the mask is a value that allows aligned objects to hold any type of data: for example, if its value is 3, any type of data can be stored at locations whose addresses are multiples of 4. A mask value of 0 means an object can start on any multiple of 1 (that is, no alignment is required).

The expansion of the macro obstack_alignment_mask is an lvalue, so you can alter the mask by assignment. For example, this statement: 

          obstack_alignment_mask (obstack_ptr) = 0;

has the effect of turning off alignment processing in the specified obstack.

Note that a change in alignment mask does not take effect until after the next time an object is allocated or finished in the obstack. If you are not growing an object, you can make the new alignment mask take effect immediately by calling obstack_finish. This will finish a zero-length object and then do proper alignment for the next object.

Next: , Previous: Obstacks Data Alignment, Up: Obstacks
3.2.4.10 Obstack Chunks

Obstacks work by allocating space for themselves in large chunks, and then parceling out space in the chunks to satisfy your requests. Chunks are normally 4096 bytes long unless you specify a different chunk size. The chunk size includes 8 bytes of overhead that are not actually used for storing objects. Regardless of the specified size, longer chunks will be allocated when necessary for long objects.

The obstack library allocates chunks by calling the function obstack_chunk_alloc, which you must define. When a chunk is no longer needed because you have freed all the objects in it, the obstack library frees the chunk by calling obstack_chunk_free, which you must also define.

These two must be defined (as macros) or declared (as functions) in each source file that uses obstack_init (see Creating Obstacks). Most often they are defined as macros like this: 

     #define obstack_chunk_alloc malloc
     #define obstack_chunk_free free

Note that these are simple macros (no arguments). Macro definitions with arguments will not work! It is necessary that obstack_chunk_alloc or obstack_chunk_free, alone, expand into a function name if it is not itself a function name.

If you allocate chunks with malloc, the chunk size should be a power of 2. The default chunk size, 4096, was chosen because it is long enough to satisfy many typical requests on the obstack yet short enough not to waste too much memory in the portion of the last chunk not yet used.

— Macro: int obstack_chunk_size (struct obstack *obstack-ptr)

This returns the chunk size of the given obstack.

Since this macro expands to an lvalue, you can specify a new chunk size by assigning it a new value. Doing so does not affect the chunks already allocated, but will change the size of chunks allocated for that particular obstack in the future. It is unlikely to be useful to make the chunk size smaller, but making it larger might improve efficiency if you are allocating many objects whose size is comparable to the chunk size. Here is how to do so cleanly: 

     if (obstack_chunk_size (obstack_ptr) < new-chunk-size)
       obstack_chunk_size (obstack_ptr) = new-chunk-size;

Previous: Obstack Chunks, Up: Obstacks
3.2.4.11 Summary of Obstack Functions

Here is a summary of all the functions associated with obstacks. Each takes the address of an obstack (struct obstack *) as its first argument.

void obstack_init (struct obstack *obstack-ptr)
Initialize use of an obstack. See Creating Obstacks.
void *obstack_alloc (struct obstack *obstack-ptr, int size)
Allocate an object of size uninitialized bytes. See Allocation in an Obstack.
void *obstack_copy (struct obstack *obstack-ptr, void *address, int size)
Allocate an object of size bytes, with contents copied from address. See Allocation in an Obstack.
void *obstack_copy0 (struct obstack *obstack-ptr, void *address, int size)
Allocate an object of size+1 bytes, with size of them copied from address, followed by a null character at the end. See Allocation in an Obstack.
void obstack_free (struct obstack *obstack-ptr, void *object)
Free object (and everything allocated in the specified obstack more recently than object). See Freeing Obstack Objects.
void obstack_blank (struct obstack *obstack-ptr, int size)
Add size uninitialized bytes to a growing object. See Growing Objects.
void obstack_grow (struct obstack *obstack-ptr, void *address, int size)
Add size bytes, copied from address, to a growing object. See Growing Objects.
void obstack_grow0 (struct obstack *obstack-ptr, void *address, int size)
Add size bytes, copied from address, to a growing object, and then add another byte containing a null character. See Growing Objects.
void obstack_1grow (struct obstack *obstack-ptr, char data-char)
Add one byte containing data-char to a growing object. See Growing Objects.
void *obstack_finish (struct obstack *obstack-ptr)
Finalize the object that is growing and return its permanent address. See Growing Objects.
int obstack_object_size (struct obstack *obstack-ptr)
Get the current size of the currently growing object. See Growing Objects.
void obstack_blank_fast (struct obstack *obstack-ptr, int size)
Add size uninitialized bytes to a growing object without checking that there is enough room. See Extra Fast Growing.
void obstack_1grow_fast (struct obstack *obstack-ptr, char data-char)
Add one byte containing data-char to a growing object without checking that there is enough room. See Extra Fast Growing.
int obstack_room (struct obstack *obstack-ptr)
Get the amount of room now available for growing the current object. See Extra Fast Growing.
int obstack_alignment_mask (struct obstack *obstack-ptr)
The mask used for aligning the beginning of an object. This is an lvalue. See Obstacks Data Alignment.
int obstack_chunk_size (struct obstack *obstack-ptr)
The size for allocating chunks. This is an lvalue. See Obstack Chunks.
void *obstack_base (struct obstack *obstack-ptr)
Tentative starting address of the currently growing object. See Status of an Obstack.
void *obstack_next_free (struct obstack *obstack-ptr)
Address just after the end of the currently growing object. See Status of an Obstack.

Previous: Obstacks, Up: Memory Allocation

3.2.5 Automatic Storage with Variable Size

The function alloca supports a kind of half-dynamic allocation in which blocks are allocated dynamically but freed automatically.

Allocating a block with alloca is an explicit action; you can allocate as many blocks as you wish, and compute the size at run time. But all the blocks are freed when you exit the function that alloca was called from, just as if they were automatic variables declared in that function. There is no way to free the space explicitly.

The prototype for alloca is in stdlib.h. This function is a BSD extension.

— Function: void * alloca (size_t size);

The return value of alloca is the address of a block of size bytes of memory, allocated in the stack frame of the calling function.

Do not use alloca inside the arguments of a function call—you will get unpredictable results, because the stack space for the alloca would appear on the stack in the middle of the space for the function arguments. An example of what to avoid is foo (x, alloca (4), y).

Next: , Up: Variable Size Automatic
3.2.5.1 alloca Example

As an example of the use of alloca, here is a function that opens a file name made from concatenating two argument strings, and returns a file descriptor or minus one signifying failure: 

     int
     open2 (char *str1, char *str2, int flags, int mode)
     {
       char *name = (char *) alloca (strlen (str1) + strlen (str2) + 1);
       stpcpy (stpcpy (name, str1), str2);
       return open (name, flags, mode);
     }

Here is how you would get the same results with malloc and free: 

     int
     open2 (char *str1, char *str2, int flags, int mode)
     {
       char *name = (char *) malloc (strlen (str1) + strlen (str2) + 1);
       int desc;
       if (name == 0)
         fatal ("virtual memory exceeded");
       stpcpy (stpcpy (name, str1), str2);
       desc = open (name, flags, mode);
       free (name);
       return desc;
     }

As you can see, it is simpler with alloca. But alloca has other, more important advantages, and some disadvantages.

Next: , Previous: Alloca Example, Up: Variable Size Automatic
3.2.5.2 Advantages of alloca

Here are the reasons why alloca may be preferable to malloc:

Next: , Previous: Advantages of Alloca, Up: Variable Size Automatic
3.2.5.3 Disadvantages of alloca

These are the disadvantages of alloca in comparison with malloc:

Previous: Disadvantages of Alloca, Up: Variable Size Automatic
3.2.5.4 GNU C Variable-Size Arrays

In GNU C, you can replace most uses of alloca with an array of variable size. Here is how open2 would look then: 

     int open2 (char *str1, char *str2, int flags, int mode)
     {
       char name[strlen (str1) + strlen (str2) + 1];
       stpcpy (stpcpy (name, str1), str2);
       return open (name, flags, mode);
     }

But alloca is not always equivalent to a variable-sized array, for several reasons:

NB: If you mix use of alloca and variable-sized arrays within one function, exiting a scope in which a variable-sized array was declared frees all blocks allocated with alloca during the execution of that scope.

Previous: Locking Pages, Up: Memory

3.3 Resizing the Data Segment

The symbols in this section are declared in unistd.h.

You will not normally use the functions in this section, because the functions described in Memory Allocation are easier to use. Those are interfaces to a GNU C Library memory allocator that uses the functions below itself. The functions below are simple interfaces to system calls.

— Function: int brk (void *addr)

brk sets the high end of the calling process' data segment to addr.

The address of the end of a segment is defined to be the address of the last byte in the segment plus 1.

The function has no effect if addr is lower than the low end of the data segment. (This is considered success, by the way).

The function fails if it would cause the data segment to overlap another segment or exceed the process' data storage limit (see Limits on Resources).

The function is named for a common historical case where data storage and the stack are in the same segment. Data storage allocation grows upward from the bottom of the segment while the stack grows downward toward it from the top of the segment and the curtain between them is called the break.

The return value is zero on success. On failure, the return value is -1 and errno is set accordingly. The following errno values are specific to this function:

ENOMEM
The request would cause the data segment to overlap another segment or exceed the process' data storage limit.
— Function: void *sbrk (ptrdiff_t delta)

This function is the same as brk except that you specify the new end of the data segment as an offset delta from the current end and on success the return value is the address of the resulting end of the data segment instead of zero.

This means you can use ‘sbrk(0)’ to find out what the current end of the data segment is.

Next: , Previous: Memory Allocation, Up: Memory

3.4 Locking Pages

You can tell the system to associate a particular virtual memory page with a real page frame and keep it that way — i.e., cause the page to be paged in if it isn't already and mark it so it will never be paged out and consequently will never cause a page fault. This is called locking a page.

The functions in this chapter lock and unlock the calling process' pages.

Next: , Up: Locking Pages

3.4.1 Why Lock Pages

Because page faults cause paged out pages to be paged in transparently, a process rarely needs to be concerned about locking pages. However, there are two reasons people sometimes are:

Be aware that when you lock a page, that's one fewer page frame that can be used to back other virtual memory (by the same or other processes), which can mean more page faults, which means the system runs more slowly. In fact, if you lock enough memory, some programs may not be able to run at all for lack of real memory.

Next: , Previous: Why Lock Pages, Up: Locking Pages

3.4.2 Locked Memory Details

A memory lock is associated with a virtual page, not a real frame. The paging rule is: If a frame backs at least one locked page, don't page it out.

Memory locks do not stack. I.e., you can't lock a particular page twice so that it has to be unlocked twice before it is truly unlocked. It is either locked or it isn't.

A memory lock persists until the process that owns the memory explicitly unlocks it. (But process termination and exec cause the virtual memory to cease to exist, which you might say means it isn't locked any more).

Memory locks are not inherited by child processes. (But note that on a modern Unix system, immediately after a fork, the parent's and the child's virtual address space are backed by the same real page frames, so the child enjoys the parent's locks). See Creating a Process.

Because of its ability to impact other processes, only the superuser can lock a page. Any process can unlock its own page.

The system sets limits on the amount of memory a process can have locked and the amount of real memory it can have dedicated to it. See Limits on Resources.

In Linux, locked pages aren't as locked as you might think. Two virtual pages that are not shared memory can nonetheless be backed by the same real frame. The kernel does this in the name of efficiency when it knows both virtual pages contain identical data, and does it even if one or both of the virtual pages are locked.

But when a process modifies one of those pages, the kernel must get it a separate frame and fill it with the page's data. This is known as a copy-on-write page fault. It takes a small amount of time and in a pathological case, getting that frame may require I/O. To make sure this doesn't happen to your program, don't just lock the pages. Write to them as well, unless you know you won't write to them ever. And to make sure you have pre-allocated frames for your stack, enter a scope that declares a C automatic variable larger than the maximum stack size you will need, set it to something, then return from its scope.

Previous: Locked Memory Details, Up: Locking Pages

3.4.3 Functions To Lock And Unlock Pages

The symbols in this section are declared in sys/mman.h. These functions are defined by POSIX.1b, but their availability depends on your kernel. If your kernel doesn't allow these functions, they exist but always fail. They are available with a Linux kernel.

Portability Note: POSIX.1b requires that when the mlock and munlock functions are available, the file unistd.h define the macro _POSIX_MEMLOCK_RANGE and the file limits.h define the macro PAGESIZE to be the size of a memory page in bytes. It requires that when the mlockall and munlockall functions are available, the unistd.h file define the macro _POSIX_MEMLOCK. The GNU C library conforms to this requirement.

— Function: int mlock (const void *addr, size_t len)

mlock locks a range of the calling process' virtual pages.

The range of memory starts at address addr and is len bytes long. Actually, since you must lock whole pages, it is the range of pages that include any part of the specified range.

When the function returns successfully, each of those pages is backed by (connected to) a real frame (is resident) and is marked to stay that way. This means the function may cause page-ins and have to wait for them.

When the function fails, it does not affect the lock status of any pages.

The return value is zero if the function succeeds. Otherwise, it is -1 and errno is set accordingly. errno values specific to this function are:

ENOMEM
  • At least some of the specified address range does not exist in the calling process' virtual address space.
  • The locking would cause the process to exceed its locked page limit.

EPERM
The calling process is not superuser.
EINVAL
len is not positive.
ENOSYS
The kernel does not provide mlock capability.

You can lock all a process' memory with mlockall. You unlock memory with munlock or munlockall.

To avoid all page faults in a C program, you have to use mlockall, because some of the memory a program uses is hidden from the C code, e.g. the stack and automatic variables, and you wouldn't know what address to tell mlock.

— Function: int munlock (const void *addr, size_t len)

munlock unlocks a range of the calling process' virtual pages.

munlock is the inverse of mlock and functions completely analogously to mlock, except that there is no EPERM failure.

— Function: int mlockall (int flags)

mlockall locks all the pages in a process' virtual memory address space, and/or any that are added to it in the future. This includes the pages of the code, data and stack segment, as well as shared libraries, user space kernel data, shared memory, and memory mapped files.

flags is a string of single bit flags represented by the following macros. They tell mlockall which of its functions you want. All other bits must be zero.

MCL_CURRENT
Lock all pages which currently exist in the calling process' virtual address space.
MCL_FUTURE
Set a mode such that any pages added to the process' virtual address space in the future will be locked from birth. This mode does not affect future address spaces owned by the same process so exec, which replaces a process' address space, wipes out MCL_FUTURE. See Executing a File.

When the function returns successfully, and you specified MCL_CURRENT, all of the process' pages are backed by (connected to) real frames (they are resident) and are marked to stay that way. This means the function may cause page-ins and have to wait for them.

When the process is in MCL_FUTURE mode because it successfully executed this function and specified MCL_CURRENT, any system call by the process that requires space be added to its virtual address space fails with errno = ENOMEM if locking the additional space would cause the process to exceed its locked page limit. In the case that the address space addition that can't be accommodated is stack expansion, the stack expansion fails and the kernel sends a SIGSEGV signal to the process.

When the function fails, it does not affect the lock status of any pages or the future locking mode.

The return value is zero if the function succeeds. Otherwise, it is -1 and errno is set accordingly. errno values specific to this function are:

ENOMEM
  • At least some of the specified address range does not exist in the calling process' virtual address space.
  • The locking would cause the process to exceed its locked page limit.

EPERM
The calling process is not superuser.
EINVAL
Undefined bits in flags are not zero.
ENOSYS
The kernel does not provide mlockall capability.

You can lock just specific pages with mlock. You unlock pages with munlockall and munlock.

— Function: int munlockall (void)

munlockall unlocks every page in the calling process' virtual address space and turn off MCL_FUTURE future locking mode.

The return value is zero if the function succeeds. Otherwise, it is -1 and errno is set accordingly. The only way this function can fail is for generic reasons that all functions and system calls can fail, so there are no specific errno values.

Next: , Previous: Memory, Up: Top

4 Character Handling

Programs that work with characters and strings often need to classify a character—is it alphabetic, is it a digit, is it whitespace, and so on—and perform case conversion operations on characters. The functions in the header file ctype.h are provided for this purpose. Since the choice of locale and character set can alter the classifications of particular character codes, all of these functions are affected by the current locale. (More precisely, they are affected by the locale currently selected for character classification—the LC_CTYPE category; see Locale Categories.)

The ISO C standard specifies two different sets of functions. The one set works on char type characters, the other one on wchar_t wide characters (see Extended Char Intro).

Next: , Up: Character Handling

4.1 Classification of Characters

This section explains the library functions for classifying characters. For example, isalpha is the function to test for an alphabetic character. It takes one argument, the character to test, and returns a nonzero integer if the character is alphabetic, and zero otherwise. You would use it like this: 

     if (isalpha (c))
       printf ("The character `%c' is alphabetic.\n", c);

Each of the functions in this section tests for membership in a particular class of characters; each has a name starting with ‘is’. Each of them takes one argument, which is a character to test, and returns an int which is treated as a boolean value. The character argument is passed as an int, and it may be the constant value EOF instead of a real character.

The attributes of any given character can vary between locales. See Locales, for more information on locales.

These functions are declared in the header file ctype.h.

— Function: int islower (int c)

Returns true if c is a lower-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.

— Function: int isupper (int c)

Returns true if c is an upper-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.

— Function: int isalpha (int c)

Returns true if c is an alphabetic character (a letter). If islower or isupper is true of a character, then isalpha is also true.

In some locales, there may be additional characters for which isalpha is true—letters which are neither upper case nor lower case. But in the standard "C" locale, there are no such additional characters.

— Function: int isdigit (int c)

Returns true if c is a decimal digit (‘0’ through ‘9’).

— Function: int isalnum (int c)

Returns true if c is an alphanumeric character (a letter or number); in other words, if either isalpha or isdigit is true of a character, then isalnum is also true.

— Function: int isxdigit (int c)

Returns true if c is a hexadecimal digit. Hexadecimal digits include the normal decimal digits ‘0’ through ‘9’ and the letters ‘A’ through ‘F’ and ‘a’ through ‘f’.

— Function: int ispunct (int c)

Returns true if c is a punctuation character. This means any printing character that is not alphanumeric or a space character.

— Function: int isspace (int c)

Returns true if c is a whitespace character. In the standard "C" locale, isspace returns true for only the standard whitespace characters:

' '
space
'\f'
formfeed
'\n'
newline
'\r'
carriage return
'\t'
horizontal tab
'\v'
vertical tab

— Function: int isblank (int c)

Returns true if c is a blank character; that is, a space or a tab. This function was originally a GNU extension, but was added in ISO C99.

— Function: int isgraph (int c)

Returns true if c is a graphic character; that is, a character that has a glyph associated with it. The whitespace characters are not considered graphic.

— Function: int isprint (int c)

Returns true if c is a printing character. Printing characters include all the graphic characters, plus the space (‘ ’) character.

— Function: int iscntrl (int c)

Returns true if c is a control character (that is, a character that is not a printing character).

— Function: int isascii (int c)

Returns true if c is a 7-bit unsigned char value that fits into the US/UK ASCII character set. This function is a BSD extension and is also an SVID extension.

Next: , Previous: Classification of Characters, Up: Character Handling

4.2 Case Conversion

This section explains the library functions for performing conversions such as case mappings on characters. For example, toupper converts any character to upper case if possible. If the character can't be converted, toupper returns it unchanged.

These functions take one argument of type int, which is the character to convert, and return the converted character as an int. If the conversion is not applicable to the argument given, the argument is returned unchanged.

Compatibility Note: In pre-ISO C dialects, instead of returning the argument unchanged, these functions may fail when the argument is not suitable for the conversion. Thus for portability, you may need to write islower(c) ? toupper(c) : c rather than just toupper(c).

These functions are declared in the header file ctype.h.

— Function: int tolower (int c)

If c is an upper-case letter, tolower returns the corresponding lower-case letter. If c is not an upper-case letter, c is returned unchanged.

— Function: int toupper (int c)

If c is a lower-case letter, toupper returns the corresponding upper-case letter. Otherwise c is returned unchanged.

— Function: int toascii (int c)

This function converts c to a 7-bit unsigned char value that fits into the US/UK ASCII character set, by clearing the high-order bits. This function is a BSD extension and is also an SVID extension.

— Function: int _tolower (int c)

This is identical to tolower, and is provided for compatibility with the SVID. See SVID.

— Function: int _toupper (int c)

This is identical to toupper, and is provided for compatibility with the SVID.

Next: , Previous: Case Conversion, Up: Character Handling

4.3 Character class determination for wide characters

Amendment 1 to ISO C90 defines functions to classify wide characters. Although the original ISO C90 standard already defined the type wchar_t, no functions operating on them were defined.

The general design of the classification functions for wide characters is more general. It allows extensions to the set of available classifications, beyond those which are always available. The POSIX standard specifies how extensions can be made, and this is already implemented in the GNU C library implementation of the localedef program.

The character class functions are normally implemented with bitsets, with a bitset per character. For a given character, the appropriate bitset is read from a table and a test is performed as to whether a certain bit is set. Which bit is tested for is determined by the class.

For the wide character classification functions this is made visible. There is a type classification type defined, a function to retrieve this value for a given class, and a function to test whether a given character is in this class, using the classification value. On top of this the normal character classification functions as used for char objects can be defined.

— Data type: wctype_t

The wctype_t can hold a value which represents a character class. The only defined way to generate such a value is by using the wctype function.

This type is defined in wctype.h.

— Function: wctype_t wctype (const char *property)

The wctype returns a value representing a class of wide characters which is identified by the string property. Beside some standard properties each locale can define its own ones. In case no property with the given name is known for the current locale selected for the LC_CTYPE category, the function returns zero.

The properties known in every locale are:

"alnum" "alpha" "cntrl" "digit"
"graph" "lower" "print" "punct"
"space" "upper" "xdigit"

This function is declared in wctype.h.

To test the membership of a character to one of the non-standard classes the ISO C standard defines a completely new function.

— Function: int iswctype (wint_t wc, wctype_t desc)

This function returns a nonzero value if wc is in the character class specified by desc. desc must previously be returned by a successful call to wctype.

This function is declared in wctype.h.

To make it easier to use the commonly-used classification functions, they are defined in the C library. There is no need to use wctype if the property string is one of the known character classes. In some situations it is desirable to construct the property strings, and then it is important that wctype can also handle the standard classes.

— Function: int iswalnum (wint_t wc)

This function returns a nonzero value if wc is an alphanumeric character (a letter or number); in other words, if either iswalpha or iswdigit is true of a character, then iswalnum is also true.

This function can be implemented using 

          iswctype (wc, wctype ("alnum"))

It is declared in wctype.h.

— Function: int iswalpha (wint_t wc)

Returns true if wc is an alphabetic character (a letter). If iswlower or iswupper is true of a character, then iswalpha is also true.

In some locales, there may be additional characters for which iswalpha is true—letters which are neither upper case nor lower case. But in the standard "C" locale, there are no such additional characters.

This function can be implemented using 

          iswctype (wc, wctype ("alpha"))

It is declared in wctype.h.

— Function: int iswcntrl (wint_t wc)

Returns true if wc is a control character (that is, a character that is not a printing character).

This function can be implemented using 

          iswctype (wc, wctype ("cntrl"))

It is declared in wctype.h.

— Function: int iswdigit (wint_t wc)

Returns true if wc is a digit (e.g., ‘0’ through ‘9’). Please note that this function does not only return a nonzero value for decimal digits, but for all kinds of digits. A consequence is that code like the following will not work unconditionally for wide characters: 

          n = 0;
          while (iswdigit (*wc))
            {
              n *= 10;
              n += *wc++ - L'0';
            }

This function can be implemented using 

          iswctype (wc, wctype ("digit"))

It is declared in wctype.h.

— Function: int iswgraph (wint_t wc)

Returns true if wc is a graphic character; that is, a character that has a glyph associated with it. The whitespace characters are not considered graphic.

This function can be implemented using 

          iswctype (wc, wctype ("graph"))

It is declared in wctype.h.

— Function: int iswlower (wint_t wc)

Returns true if wc is a lower-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.

This function can be implemented using 

          iswctype (wc, wctype ("lower"))

It is declared in wctype.h.

— Function: int iswprint (wint_t wc)

Returns true if wc is a printing character. Printing characters include all the graphic characters, plus the space (‘ ’) character.

This function can be implemented using 

          iswctype (wc, wctype ("print"))

It is declared in wctype.h.

— Function: int iswpunct (wint_t wc)

Returns true if wc is a punctuation character. This means any printing character that is not alphanumeric or a space character.

This function can be implemented using 

          iswctype (wc, wctype ("punct"))

It is declared in wctype.h.

— Function: int iswspace (wint_t wc)

Returns true if wc is a whitespace character. In the standard "C" locale, iswspace returns true for only the standard whitespace characters:

L' '
space
L'\f'
formfeed
L'\n'
newline
L'\r'
carriage return
L'\t'
horizontal tab
L'\v'
vertical tab

This function can be implemented using 

          iswctype (wc, wctype ("space"))

It is declared in wctype.h.

— Function: int iswupper (wint_t wc)

Returns true if wc is an upper-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid.

This function can be implemented using 

          iswctype (wc, wctype ("upper"))

It is declared in wctype.h.

— Function: int iswxdigit (wint_t wc)

Returns true if wc is a hexadecimal digit. Hexadecimal digits include the normal decimal digits ‘0’ through ‘9’ and the letters ‘A’ through ‘F’ and ‘a’ through ‘f’.

This function can be implemented using 

          iswctype (wc, wctype ("xdigit"))

It is declared in wctype.h.

The GNU C library also provides a function which is not defined in the ISO C standard but which is available as a version for single byte characters as well.

— Function: int iswblank (wint_t wc)

Returns true if wc is a blank character; that is, a space or a tab. This function was originally a GNU extension, but was added in ISO C99. It is declared in wchar.h.

Next: , Previous: Classification of Wide Characters, Up: Character Handling

4.4 Notes on using the wide character classes

The first note is probably not astonishing but still occasionally a cause of problems. The iswXXX functions can be implemented using macros and in fact, the GNU C library does this. They are still available as real functions but when the wctype.h header is included the macros will be used. This is the same as the char type versions of these functions.

The second note covers something new. It can be best illustrated by a (real-world) example. The first piece of code is an excerpt from the original code. It is truncated a bit but the intention should be clear. 

     int
     is_in_class (int c, const char *class)
     {
       if (strcmp (class, "alnum") == 0)
         return isalnum (c);
       if (strcmp (class, "alpha") == 0)
         return isalpha (c);
       if (strcmp (class, "cntrl") == 0)
         return iscntrl (c);
       ...
       return 0;
     }

Now, with the wctype and iswctype you can avoid the if cascades, but rewriting the code as follows is wrong: 

     int
     is_in_class (int c, const char *class)
     {
       wctype_t desc = wctype (class);
       return desc ? iswctype ((wint_t) c, desc) : 0;
     }

The problem is that it is not guaranteed that the wide character representation of a single-byte character can be found using casting. In fact, usually this fails miserably. The correct solution to this problem is to write the code as follows: 

     int
     is_in_class (int c, const char *class)
     {
       wctype_t desc = wctype (class);
       return desc ? iswctype (btowc (c), desc) : 0;
     }

See Converting a Character, for more information on btowc. Note that this change probably does not improve the performance of the program a lot since the wctype function still has to make the string comparisons. It gets really interesting if the is_in_class function is called more than once for the same class name. In this case the variable desc could be computed once and reused for all the calls. Therefore the above form of the function is probably not the final one.

Previous: Using Wide Char Classes, Up: Character Handling

4.5 Mapping of wide characters.

The classification functions are also generalized by the ISO C standard. Instead of just allowing the two standard mappings, a locale can contain others. Again, the localedef program already supports generating such locale data files.

— Data Type: wctrans_t

This data type is defined as a scalar type which can hold a value representing the locale-dependent character mapping. There is no way to construct such a value apart from using the return value of the wctrans function.

This type is defined in wctype.h.

— Function: wctrans_t wctrans (const char *property)

The wctrans function has to be used to find out whether a named mapping is defined in the current locale selected for the LC_CTYPE category. If the returned value is non-zero, you can use it afterwards in calls to towctrans. If the return value is zero no such mapping is known in the current locale.

Beside locale-specific mappings there are two mappings which are guaranteed to be available in every locale:

"tolower" "toupper"

These functions are declared in wctype.h.

— Function: wint_t towctrans (wint_t wc, wctrans_t desc)

towctrans maps the input character wc according to the rules of the mapping for which desc is a descriptor, and returns the value it finds. desc must be obtained by a successful call to wctrans.

This function is declared in wctype.h.

For the generally available mappings, the ISO C standard defines convenient shortcuts so that it is not necessary to call wctrans for them.

— Function: wint_t towlower (wint_t wc)

If wc is an upper-case letter, towlower returns the corresponding lower-case letter. If wc is not an upper-case letter, wc is returned unchanged.

towlower can be implemented using 

          towctrans (wc, wctrans ("tolower"))

This function is declared in wctype.h.

— Function: wint_t towupper (wint_t wc)

If wc is a lower-case letter, towupper returns the corresponding upper-case letter. Otherwise wc is returned unchanged.

towupper can be implemented using 

          towctrans (wc, wctrans ("toupper"))

This function is declared in wctype.h.

The same warnings given in the last section for the use of the wide character classification functions apply here. It is not possible to simply cast a char type value to a wint_t and use it as an argument to towctrans calls.

Next: , Previous: Character Handling, Up: Top

5 String and Array Utilities

Operations on strings (or arrays of characters) are an important part of many programs. The GNU C library provides an extensive set of string utility functions, including functions for copying, concatenating, comparing, and searching strings. Many of these functions can also operate on arbitrary regions of storage; for example, the memcpy function can be used to copy the contents of any kind of array.

It's fairly common for beginning C programmers to “reinvent the wheel” by duplicating this functionality in their own code, but it pays to become familiar with the library functions and to make use of them, since this offers benefits in maintenance, efficiency, and portability.

For instance, you could easily compare one string to another in two lines of C code, but if you use the built-in strcmp function, you're less likely to make a mistake. And, since these library functions are typically highly optimized, your program may run faster too.

Next: , Up: String and Array Utilities

5.1 Representation of Strings

This section is a quick summary of string concepts for beginning C programmers. It describes how character strings are represented in C and some common pitfalls. If you are already familiar with this material, you can skip this section.

A string is an array of char objects. But string-valued variables are usually declared to be pointers of type char *. Such variables do not include space for the text of a string; that has to be stored somewhere else—in an array variable, a string constant, or dynamically allocated memory (see Memory Allocation). It's up to you to store the address of the chosen memory space into the pointer variable. Alternatively you can store a null pointer in the pointer variable. The null pointer does not point anywhere, so attempting to reference the string it points to gets an error.

“string” normally refers to multibyte character strings as opposed to wide character strings. Wide character strings are arrays of type wchar_t and as for multibyte character strings usually pointers of type wchar_t * are used.

By convention, a null character, '\0', marks the end of a multibyte character string and the null wide character, L'\0', marks the end of a wide character string. For example, in testing to see whether the char * variable p points to a null character marking the end of a string, you can write !*p or *p == '\0'.

A null character is quite different conceptually from a null pointer, although both are represented by the integer 0.

String literals appear in C program source as strings of characters between double-quote characters (‘"’) where the initial double-quote character is immediately preceded by a capital ‘L’ (ell) character (as in L"foo"). In ISO C, string literals can also be formed by string concatenation: "a" "b" is the same as "ab". For wide character strings one can either use L"a" L"b" or L"a" "b". Modification of string literals is not allowed by the GNU C compiler, because literals are placed in read-only storage.

Character arrays that are declared const cannot be modified either. It's generally good style to declare non-modifiable string pointers to be of type const char *, since this often allows the C compiler to detect accidental modifications as well as providing some amount of documentation about what your program intends to do with the string.

The amount of memory allocated for the character array may extend past the null character that normally marks the end of the string. In this document, the term allocated size is always used to refer to the total amount of memory allocated for the string, while the term length refers to the number of characters up to (but not including) the terminating null character. A notorious source of program bugs is trying to put more characters in a string than fit in its allocated size. When writing code that extends strings or moves characters into a pre-allocated array, you should be very careful to keep track of the length of the text and make explicit checks for overflowing the array. Many of the library functions do not do this for you! Remember also that you need to allocate an extra byte to hold the null character that marks the end of the string.

Originally strings were sequences of bytes where each byte represents a single character. This is still true today if the strings are encoded using a single-byte character encoding. Things are different if the strings are encoded using a multibyte encoding (for more information on encodings see Extended Char Intro). There is no difference in the programming interface for these two kind of strings; the programmer has to be aware of this and interpret the byte sequences accordingly.

But since there is no separate interface taking care of these differences the byte-based string functions are sometimes hard to use. Since the count parameters of these functions specify bytes a call to strncpy could cut a multibyte character in the middle and put an incomplete (and therefore unusable) byte sequence in the target buffer.

To avoid these problems later versions of the ISO C standard introduce a second set of functions which are operating on wide characters (see Extended Char Intro). These functions don't have the problems the single-byte versions have since every wide character is a legal, interpretable value. This does not mean that cutting wide character strings at arbitrary points is without problems. It normally is for alphabet-based languages (except for non-normalized text) but languages based on syllables still have the problem that more than one wide character is necessary to complete a logical unit. This is a higher level problem which the C library functions are not designed to solve. But it is at least good that no invalid byte sequences can be created. Also, the higher level functions can also much easier operate on wide character than on multibyte characters so that a general advise is to use wide characters internally whenever text is more than simply copied.

The remaining of this chapter will discuss the functions for handling wide character strings in parallel with the discussion of the multibyte character strings since there is almost always an exact equivalent available.

Next: , Previous: Representation of Strings, Up: String and Array Utilities

5.2 String and Array Conventions

This chapter describes both functions that work on arbitrary arrays or blocks of memory, and functions that are specific to null-terminated arrays of characters and wide characters.

Functions that operate on arbitrary blocks of memory have names beginning with ‘mem’ and ‘wmem’ (such as memcpy and wmemcpy) and invariably take an argument which specifies the size (in bytes and wide characters respectively) of the block of memory to operate on. The array arguments and return values for these functions have type void * or wchar_t. As a matter of style, the elements of the arrays used with the ‘mem’ functions are referred to as “bytes”. You can pass any kind of pointer to these functions, and the sizeof operator is useful in computing the value for the size argument. Parameters to the ‘wmem’ functions must be of type wchar_t *. These functions are not really usable with anything but arrays of this type.

In contrast, functions that operate specifically on strings and wide character strings have names beginning with ‘str’ and ‘wcs’ respectively (such as strcpy and wcscpy) and look for a null character to terminate the string instead of requiring an explicit size argument to be passed. (Some of these functions accept a specified maximum length, but they also check for premature termination with a null character.) The array arguments and return values for these functions have type char * and wchar_t * respectively, and the array elements are referred to as “characters” and “wide characters”.

In many cases, there are both ‘mem’ and ‘str’/‘wcs’ versions of a function. The one that is more appropriate to use depends on the exact situation. When your program is manipulating arbitrary arrays or blocks of storage, then you should always use the ‘mem’ functions. On the other hand, when you are manipulating null-terminated strings it is usually more convenient to use the ‘str’/‘wcs’ functions, unless you already know the length of the string in advance. The ‘wmem’ functions should be used for wide character arrays with known size.

Some of the memory and string functions take single characters as arguments. Since a value of type char is automatically promoted into an value of type int when used as a parameter, the functions are declared with int as the type of the parameter in question. In case of the wide character function the situation is similarly: the parameter type for a single wide character is wint_t and not wchar_t. This would for many implementations not be necessary since the wchar_t is large enough to not be automatically promoted, but since the ISO C standard does not require such a choice of types the wint_t type is used.

Next: , Previous: String/Array Conventions, Up: String and Array Utilities

5.3 String Length

You can get the length of a string using the strlen function. This function is declared in the header file string.h.

— Function: size_t strlen (const char *s)

The strlen function returns the length of the null-terminated string s in bytes. (In other words, it returns the offset of the terminating null character within the array.)

For example, 

          strlen ("hello, world")
              ⇒ 12

When applied to a character array, the strlen function returns the length of the string stored there, not its allocated size. You can get the allocated size of the character array that holds a string using the sizeof operator: 

          char string[32] = "hello, world";
          sizeof (string)
              ⇒ 32
          strlen (string)
              ⇒ 12

But beware, this will not work unless string is the character array itself, not a pointer to it. For example: 

          char string[32] = "hello, world";
          char *ptr = string;
          sizeof (string)
              ⇒ 32
          sizeof (ptr)
              ⇒ 4  /* (on a machine with 4 byte pointers) */

This is an easy mistake to make when you are working with functions that take string arguments; those arguments are always pointers, not arrays.

It must also be noted that for multibyte encoded strings the return value does not have to correspond to the number of characters in the string. To get this value the string can be converted to wide characters and wcslen can be used or something like the following code can be used: 

          /* The input is in string.
             The length is expected in n.  */
          {
            mbstate_t t;
            char *scopy = string;
            /* In initial state.  */
            memset (&t, '\0', sizeof (t));
            /* Determine number of characters.  */
            n = mbsrtowcs (NULL, &scopy, strlen (scopy), &t);
          }

This is cumbersome to do so if the number of characters (as opposed to bytes) is needed often it is better to work with wide characters.

The wide character equivalent is declared in wchar.h.

— Function: size_t wcslen (const wchar_t *ws)

The wcslen function is the wide character equivalent to strlen. The return value is the number of wide characters in the wide character string pointed to by ws (this is also the offset of the terminating null wide character of ws).

Since there are no multi wide character sequences making up one character the return value is not only the offset in the array, it is also the number of wide characters.

This function was introduced in Amendment 1 to ISO C90.

— Function: size_t strnlen (const char *s, size_t maxlen)

The strnlen function returns the length of the string s in bytes if this length is smaller than maxlen bytes. Otherwise it returns maxlen. Therefore this function is equivalent to (strlen (s) < n ? strlen (s) : maxlen) but it is more efficient and works even if the string s is not null-terminated. 

          char string[32] = "hello, world";
          strnlen (string, 32)
              ⇒ 12
          strnlen (string, 5)
              ⇒ 5

This function is a GNU extension and is declared in string.h.

— Function: size_t wcsnlen (const wchar_t *ws, size_t maxlen)

wcsnlen is the wide character equivalent to strnlen. The maxlen parameter specifies the maximum number of wide characters.

This function is a GNU extension and is declared in wchar.h.

Next: , Previous: String Length, Up: String and Array Utilities

5.4 Copying and Concatenation

You can use the functions described in this section to copy the contents of strings and arrays, or to append the contents of one string to another. The ‘str’ and ‘mem’ functions are declared in the header file string.h while the ‘wstr’ and ‘wmem’ functions are declared in the file wchar.h. A helpful way to remember the ordering of the arguments to the functions in this section is that it corresponds to an assignment expression, with the destination array specified to the left of the source array. All of these functions return the address of the destination array.

Most of these functions do not work properly if the source and destination arrays overlap. For example, if the beginning of the destination array overlaps the end of the source array, the original contents of that part of the source array may get overwritten before it is copied. Even worse, in the case of the string functions, the null character marking the end of the string may be lost, and the copy function might get stuck in a loop trashing all the memory allocated to your program.

All functions that have problems copying between overlapping arrays are explicitly identified in this manual. In addition to functions in this section, there are a few others like sprintf (see Formatted Output Functions) and scanf (see Formatted Input Functions).

— Function: void * memcpy (void *restrict to, const void *restrict from, size_t size)

The memcpy function copies size bytes from the object beginning at from into the object beginning at to. The behavior of this function is undefined if the two arrays to and from overlap; use memmove instead if overlapping is possible.

The value returned by memcpy is the value of to.

Here is an example of how you might use memcpy to copy the contents of an array: 

          struct foo *oldarray, *newarray;
          int arraysize;
          ...
          memcpy (new, old, arraysize * sizeof (struct foo));
— Function: wchar_t * wmemcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)

The wmemcpy function copies size wide characters from the object beginning at wfrom into the object beginning at wto. The behavior of this function is undefined if the two arrays wto and wfrom overlap; use wmemmove instead if overlapping is possible.

The following is a possible implementation of wmemcpy but there are more optimizations possible. 

          wchar_t *
          wmemcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom,
                   size_t size)
          {
            return (wchar_t *) memcpy (wto, wfrom, size * sizeof (wchar_t));
          }

The value returned by wmemcpy is the value of wto.

This function was introduced in Amendment 1 to ISO C90.

— Function: void * mempcpy (void *restrict to, const void *restrict from, size_t size)

The mempcpy function is nearly identical to the memcpy function. It copies size bytes from the object beginning at from into the object pointed to by to. But instead of returning the value of to it returns a pointer to the byte following the last written byte in the object beginning at to. I.e., the value is ((void *) ((char *) to + size)).

This function is useful in situations where a number of objects shall be copied to consecutive memory positions. 

          void *
          combine (void *o1, size_t s1, void *o2, size_t s2)
          {
            void *result = malloc (s1 + s2);
            if (result != NULL)
              mempcpy (mempcpy (result, o1, s1), o2, s2);
            return result;
          }

This function is a GNU extension.

— Function: wchar_t * wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)

The wmempcpy function is nearly identical to the wmemcpy function. It copies size wide characters from the object beginning at wfrom into the object pointed to by wto. But instead of returning the value of wto it returns a pointer to the wide character following the last written wide character in the object beginning at wto. I.e., the value is wto + size.

This function is useful in situations where a number of objects shall be copied to consecutive memory positions.

The following is a possible implementation of wmemcpy but there are more optimizations possible. 

          wchar_t *
          wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom,
                    size_t size)
          {
            return (wchar_t *) mempcpy (wto, wfrom, size * sizeof (wchar_t));
          }

This function is a GNU extension.

— Function: void * memmove (void *to, const void *from, size_t size)

memmove copies the size bytes at from into the size bytes at to, even if those two blocks of space overlap. In the case of overlap, memmove is careful to copy the original values of the bytes in the block at from, including those bytes which also belong to the block at to.

The value returned by memmove is the value of to.

— Function: wchar_t * wmemmove (wchar *wto, const wchar_t *wfrom, size_t size)

wmemmove copies the size wide characters at wfrom into the size wide characters at wto, even if those two blocks of space overlap. In the case of overlap, memmove is careful to copy the original values of the wide characters in the block at wfrom, including those wide characters which also belong to the block at wto.

The following is a possible implementation of wmemcpy but there are more optimizations possible. 

          wchar_t *
          wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom,
                    size_t size)
          {
            return (wchar_t *) mempcpy (wto, wfrom, size * sizeof (wchar_t));
          }

The value returned by wmemmove is the value of wto.

This function is a GNU extension.

— Function: void * memccpy (void *restrict to, const void *restrict from, int c, size_t size)

This function copies no more than size bytes from from to to, stopping if a byte matching c is found. The return value is a pointer into to one byte past where c was copied, or a null pointer if no byte matching c appeared in the first size bytes of from.

— Function: void * memset (void *block, int c, size_t size)

This function copies the value of c (converted to an unsigned char) into each of the first size bytes of the object beginning at block. It returns the value of block.

— Function: wchar_t * wmemset (wchar_t *block, wchar_t wc, size_t size)

This function copies the value of wc into each of the first size wide characters of the object beginning at block. It returns the value of block.

— Function: char * strcpy (char *restrict to, const char *restrict from)

This copies characters from the string from (up to and including the terminating null character) into the string to. Like memcpy, this function has undefined results if the strings overlap. The return value is the value of to.

— Function: wchar_t * wcscpy (wchar_t *restrict wto, const wchar_t *restrict wfrom)

This copies wide characters from the string wfrom (up to and including the terminating null wide character) into the string wto. Like wmemcpy, this function has undefined results if the strings overlap. The return value is the value of wto.

— Function: char * strncpy (char *restrict to, const char *restrict from, size_t size)

This function is similar to strcpy but always copies exactly size characters into to.

If the length of from is more than size, then strncpy copies just the first size characters. Note that in this case there is no null terminator written into to.

If the length of from is less than size, then strncpy copies all of from, followed by enough null characters to add up to size characters in all. This behavior is rarely useful, but it is specified by the ISO C standard.

The behavior of strncpy is undefined if the strings overlap.

Using strncpy as opposed to strcpy is a way to avoid bugs relating to writing past the end of the allocated space for to. However, it can also make your program much slower in one common case: copying a string which is probably small into a potentially large buffer. In this case, size may be large, and when it is, strncpy will waste a considerable amount of time copying null characters.

— Function: wchar_t * wcsncpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)

This function is similar to wcscpy but always copies exactly size wide characters into wto.

If the length of wfrom is more than size, then wcsncpy copies just the first size wide characters. Note that in this case there is no null terminator written into wto.

If the length of wfrom is less than size, then wcsncpy copies all of wfrom, followed by enough null wide characters to add up to size wide characters in all. This behavior is rarely useful, but it is specified by the ISO C standard.

The behavior of wcsncpy is undefined if the strings overlap.

Using wcsncpy as opposed to wcscpy is a way to avoid bugs relating to writing past the end of the allocated space for wto. However, it can also make your program much slower in one common case: copying a string which is probably small into a potentially large buffer. In this case, size may be large, and when it is, wcsncpy will waste a considerable amount of time copying null wide characters.

— Function: char * strdup (const char *s)

This function copies the null-terminated string s into a newly allocated string. The string is allocated using malloc; see Unconstrained Allocation. If malloc cannot allocate space for the new string, strdup returns a null pointer. Otherwise it returns a pointer to the new string.

— Function: wchar_t * wcsdup (const wchar_t *ws)

This function copies the null-terminated wide character string ws into a newly allocated string. The string is allocated using malloc; see Unconstrained Allocation. If malloc cannot allocate space for the new string, wcsdup returns a null pointer. Otherwise it returns a pointer to the new wide character string.

This function is a GNU extension.

— Function: char * strndup (const char *s, size_t size)

This function is similar to strdup but always copies at most size characters into the newly allocated string.

If the length of s is more than size, then strndup copies just the first size characters and adds a closing null terminator. Otherwise all characters are copied and the string is terminated.

This function is different to strncpy in that it always terminates the destination string.

strndup is a GNU extension.

— Function: char * stpcpy (char *restrict to, const char *restrict from)

This function is like strcpy, except that it returns a pointer to the end of the string to (that is, the address of the terminating null character to + strlen (from)) rather than the beginning.

For example, this program uses stpcpy to concatenate ‘foo’ and ‘bar’ to produce ‘foobar’, which it then prints. 

          #include <string.h>
          #include <stdio.h>
          
          int
          main (void)
          {
            char buffer[10];
            char *to = buffer;
            to = stpcpy (to, "foo");
            to = stpcpy (to, "bar");
            puts (buffer);
            return 0;
          }

This function is not part of the ISO or POSIX standards, and is not customary on Unix systems, but we did not invent it either. Perhaps it comes from MS-DOG.

Its behavior is undefined if the strings overlap. The function is declared in string.h.

— Function: wchar_t * wcpcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom)

This function is like wcscpy, except that it returns a pointer to the end of the string wto (that is, the address of the terminating null character wto + strlen (wfrom)) rather than the beginning.

This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself.

The behavior of wcpcpy is undefined if the strings overlap.

wcpcpy is a GNU extension and is declared in wchar.h.

— Function: char * stpncpy (char *restrict to, const char *restrict from, size_t size)

This function is similar to stpcpy but copies always exactly size characters into to.

If the length of from is more then size, then stpncpy copies just the first size characters and returns a pointer to the character directly following the one which was copied last. Note that in this case there is no null terminator written into to.

If the length of from is less than size, then stpncpy copies all of from, followed by enough null characters to add up to size characters in all. This behavior is rarely useful, but it is implemented to be useful in contexts where this behavior of the strncpy is used. stpncpy returns a pointer to the first written null character.

This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself.

Its behavior is undefined if the strings overlap. The function is declared in string.h.

— Function: wchar_t * wcpncpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)

This function is similar to wcpcpy but copies always exactly wsize characters into wto.

If the length of wfrom is more then size, then wcpncpy copies just the first size wide characters and returns a pointer to the wide character directly following the last non-null wide character which was copied last. Note that in this case there is no null terminator written into wto.

If the length of wfrom is less than size, then wcpncpy copies all of wfrom, followed by enough null characters to add up to size characters in all. This behavior is rarely useful, but it is implemented to be useful in contexts where this behavior of the wcsncpy is used. wcpncpy returns a pointer to the first written null character.

This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself.

Its behavior is undefined if the strings overlap.

wcpncpy is a GNU extension and is declared in wchar.h.

— Macro: char * strdupa (const char *s)

This macro is similar to strdup but allocates the new string using alloca instead of malloc (see Variable Size Automatic). This means of course the returned string has the same limitations as any block of memory allocated using alloca.

For obvious reasons strdupa is implemented only as a macro; you cannot get the address of this function. Despite this limitation it is a useful function. The following code shows a situation where using malloc would be a lot more expensive. 

          #include <paths.h>
          #include <string.h>
          #include <stdio.h>
          
          const char path[] = _PATH_STDPATH;
          
          int
          main (void)
          {
            char *wr_path = strdupa (path);
            char *cp = strtok (wr_path, ":");
          
            while (cp != NULL)
              {
                puts (cp);
                cp = strtok (NULL, ":");
              }
            return 0;
          }

Please note that calling strtok using path directly is invalid. It is also not allowed to call strdupa in the argument list of strtok since strdupa uses alloca (see Variable Size Automatic) can interfere with the parameter passing.

This function is only available if GNU CC is used.

— Macro: char * strndupa (const char *s, size_t size)

This function is similar to strndup but like strdupa it allocates the new string using alloca see Variable Size Automatic. The same advantages and limitations of strdupa are valid for strndupa, too.

This function is implemented only as a macro, just like strdupa. Just as strdupa this macro also must not be used inside the parameter list in a function call.

strndupa is only available if GNU CC is used.

— Function: char * strcat (char *restrict to, const char *restrict from)

The strcat function is similar to strcpy, except that the characters from from are concatenated or appended to the end of to, instead of overwriting it. That is, the first character from from overwrites the null character marking the end of to.

An equivalent definition for strcat would be: 

          char *
          strcat (char *restrict to, const char *restrict from)
          {
            strcpy (to + strlen (to), from);
            return to;
          }

This function has undefined results if the strings overlap.

— Function: wchar_t * wcscat (wchar_t *restrict wto, const wchar_t *restrict wfrom)

The wcscat function is similar to wcscpy, except that the characters from wfrom are concatenated or appended to the end of wto, instead of overwriting it. That is, the first character from wfrom overwrites the null character marking the end of wto.

An equivalent definition for wcscat would be: 

          wchar_t *
          wcscat (wchar_t *wto, const wchar_t *wfrom)
          {
            wcscpy (wto + wcslen (wto), wfrom);
            return wto;
          }

This function has undefined results if the strings overlap.

Programmers using the strcat or wcscat function (or the following strncat or wcsncar functions for that matter) can easily be recognized as lazy and reckless. In almost all situations the lengths of the participating strings are known (it better should be since how can one otherwise ensure the allocated size of the buffer is sufficient?) Or at least, one could know them if one keeps track of the results of the various function calls. But then it is very inefficient to use strcat/wcscat. A lot of time is wasted finding the end of the destination string so that the actual copying can start. This is a common example: 

     /* This function concatenates arbitrarily many strings.  The last
        parameter must be NULL.  */
     char *
     concat (const char *str, ...)
     {
       va_list ap, ap2;
       size_t total = 1;
       const char *s;
       char *result;
     
       va_start (ap, str);
       /* Actually va_copy, but this is the name more gcc versions
          understand.  */
       __va_copy (ap2, ap);
     
       /* Determine how much space we need.  */
       for (s = str; s != NULL; s = va_arg (ap, const char *))
         total += strlen (s);
     
       va_end (ap);
     
       result = (char *) malloc (total);
       if (result != NULL)
         {
           result[0] = '\0';
     
           /* Copy the strings.  */
           for (s = str; s != NULL; s = va_arg (ap2, const char *))
             strcat (result, s);
         }
     
       va_end (ap2);
     
       return result;
     }

This looks quite simple, especially the second loop where the strings are actually copied. But these innocent lines hide a major performance penalty. Just imagine that ten strings of 100 bytes each have to be concatenated. For the second string we search the already stored 100 bytes for the end of the string so that we can append the next string. For all strings in total the comparisons necessary to find the end of the intermediate results sums up to 5500! If we combine the copying with the search for the allocation we can write this function more efficient: 

     char *
     concat (const char *str, ...)
     {
       va_list ap;
       size_t allocated = 100;
       char *result = (char *) malloc (allocated);
     
       if (result != NULL)
         {
           char *newp;
           char *wp;
     
           va_start (ap, str);
     
           wp = result;
           for (s = str; s != NULL; s = va_arg (ap, const char *))
             {
               size_t len = strlen (s);
     
               /* Resize the allocated memory if necessary.  */
               if (wp + len + 1 > result + allocated)
                 {
                   allocated = (allocated + len) * 2;
                   newp = (char *) realloc (result, allocated);
                   if (newp == NULL)
                     {
                       free (result);
                       return NULL;
                     }
                   wp = newp + (wp - result);
                   result = newp;
                 }
     
               wp = mempcpy (wp, s, len);
             }
     
           /* Terminate the result string.  */
           *wp++ = '\0';
     
           /* Resize memory to the optimal size.  */
           newp = realloc (result, wp - result);
           if (newp != NULL)
             result = newp;
     
           va_end (ap);
         }
     
       return result;
     }

With a bit more knowledge about the input strings one could fine-tune the memory allocation. The difference we are pointing to here is that we don't use strcat anymore. We always keep track of the length of the current intermediate result so we can safe us the search for the end of the string and use mempcpy. Please note that we also don't use stpcpy which might seem more natural since we handle with strings. But this is not necessary since we already know the length of the string and therefore can use the faster memory copying function. The example would work for wide characters the same way.

Whenever a programmer feels the need to use strcat she or he should think twice and look through the program whether the code cannot be rewritten to take advantage of already calculated results. Again: it is almost always unnecessary to use strcat.

— Function: char * strncat (char *restrict to, const char *restrict from, size_t size)

This function is like strcat except that not more than size characters from from are appended to the end of to. A single null character is also always appended to to, so the total allocated size of to must be at least size + 1 bytes longer than its initial length.

The strncat function could be implemented like this: 

          char *
          strncat (char *to, const char *from, size_t size)
          {
            to[strlen (to) + size] = '\0';
            strncpy (to + strlen (to), from, size);
            return to;
          }

The behavior of strncat is undefined if the strings overlap.

— Function: wchar_t * wcsncat (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size)

This function is like wcscat except that not more than size characters from from are appended to the end of to. A single null character is also always appended to to, so the total allocated size of to must be at least size + 1 bytes longer than its initial length.

The wcsncat function could be implemented like this: 

          wchar_t *
          wcsncat (wchar_t *restrict wto, const wchar_t *restrict wfrom,
                   size_t size)
          {
            wto[wcslen (to) + size] = L'\0';
            wcsncpy (wto + wcslen (wto), wfrom, size);
            return wto;
          }

The behavior of wcsncat is undefined if the strings overlap.

Here is an example showing the use of strncpy and strncat (the wide character version is equivalent). Notice how, in the call to strncat, the size parameter is computed to avoid overflowing the character array buffer. 

     #include <string.h>
     #include <stdio.h>
     
     #define SIZE 10
     
     static char buffer[SIZE];
     
     main ()
     {
       strncpy (buffer, "hello", SIZE);
       puts (buffer);
       strncat (buffer, ", world", SIZE - strlen (buffer) - 1);
       puts (buffer);
     }

The output produced by this program looks like: 

     hello
     hello, wo
— Function: void bcopy (const void *from, void *to, size_t size)

This is a partially obsolete alternative for memmove, derived from BSD. Note that it is not quite equivalent to memmove, because the arguments are not in the same order and there is no return value.

— Function: void bzero (void *block, size_t size)

This is a partially obsolete alternative for memset, derived from BSD. Note that it is not as general as memset, because the only value it can store is zero.

Next: , Previous: Copying and Concatenation, Up: String and Array Utilities

5.5 String/Array Comparison

You can use the functions in this section to perform comparisons on the contents of strings and arrays. As well as checking for equality, these functions can also be used as the ordering functions for sorting operations. See Searching and Sorting, for an example of this.

Unlike most comparison operations in C, the string comparison functions return a nonzero value if the strings are not equivalent rather than if they are. The sign of the value indicates the relative ordering of the first characters in the strings that are not equivalent: a negative value indicates that the first string is “less” than the second, while a positive value indicates that the first string is “greater”.

The most common use of these functions is to check only for equality. This is canonically done with an expression like ‘! strcmp (s1, s2)’.

All of these functions are declared in the header file string.h.

— Function: int memcmp (const void *a1, const void *a2, size_t size)

The function memcmp compares the size bytes of memory beginning at a1 against the size bytes of memory beginning at a2. The value returned has the same sign as the difference between the first differing pair of bytes (interpreted as unsigned char objects, then promoted to int).

If the contents of the two blocks are equal, memcmp returns 0.

— Function: int wmemcmp (const wchar_t *a1, const wchar_t *a2, size_t size)

The function wmemcmp compares the size wide characters beginning at a1 against the size wide characters beginning at a2. The value returned is smaller than or larger than zero depending on whether the first differing wide character is a1 is smaller or larger than the corresponding character in a2.

If the contents of the two blocks are equal, wmemcmp returns 0.

On arbitrary arrays, the memcmp function is mostly useful for testing equality. It usually isn't meaningful to do byte-wise ordering comparisons on arrays of things other than bytes. For example, a byte-wise comparison on the bytes that make up floating-point numbers isn't likely to tell you anything about the relationship between the values of the floating-point numbers.

wmemcmp is really only useful to compare arrays of type wchar_t since the function looks at sizeof (wchar_t) bytes at a time and this number of bytes is system dependent.

You should also be careful about using memcmp to compare objects that can contain “holes”, such as the padding inserted into structure objects to enforce alignment requirements, extra space at the end of unions, and extra characters at the ends of strings whose length is less than their allocated size. The contents of these “holes” are indeterminate and may cause strange behavior when performing byte-wise comparisons. For more predictable results, perform an explicit component-wise comparison.

For example, given a structure type definition like: 

     struct foo
       {
         unsigned char tag;
         union
           {
             double f;
             long i;
             char *p;
           } value;
       };

you are better off writing a specialized comparison function to compare struct foo objects instead of comparing them with memcmp.

— Function: int strcmp (const char *s1, const char *s2)

The strcmp function compares the string s1 against s2, returning a value that has the same sign as the difference between the first differing pair of characters (interpreted as unsigned char objects, then promoted to int).

If the two strings are equal, strcmp returns 0.

A consequence of the ordering used by strcmp is that if s1 is an initial substring of s2, then s1 is considered to be “less than” s2.

strcmp does not take sorting conventions of the language the strings are written in into account. To get that one has to use strcoll.

— Function: int wcscmp (const wchar_t *ws1, const wchar_t *ws2)

The wcscmp function compares the wide character string ws1 against ws2. The value returned is smaller than or larger than zero depending on whether the first differing wide character is ws1 is smaller or larger than the corresponding character in ws2.

If the two strings are equal, wcscmp returns 0.

A consequence of the ordering used by wcscmp is that if ws1 is an initial substring of ws2, then ws1 is considered to be “less than” ws2.

wcscmp does not take sorting conventions of the language the strings are written in into account. To get that one has to use wcscoll.

— Function: int strcasecmp (const char *s1, const char *s2)

This function is like strcmp, except that differences in case are ignored. How uppercase and lowercase characters are related is determined by the currently selected locale. In the standard "C" locale the characters Ä and ä do not match but in a locale which regards these characters as parts of the alphabet they do match.

strcasecmp is derived from BSD.

— Function: int wcscasecmp (const wchar_t *ws1, const wchar_T *ws2)

This function is like wcscmp, except that differences in case are ignored. How uppercase and lowercase characters are related is determined by the currently selected locale. In the standard "C" locale the characters Ä and ä do not match but in a locale which regards these characters as parts of the alphabet they do match.

wcscasecmp is a GNU extension.

— Function: int strncmp (const char *s1, const char *s2, size_t size)

This function is the similar to strcmp, except that no more than size characters are compared. In other words, if the two strings are the same in their first size characters, the return value is zero.

— Function: int wcsncmp (const wchar_t *ws1, const wchar_t *ws2, size_t size)

This function is the similar to wcscmp, except that no more than size wide characters are compared. In other words, if the two strings are the same in their first size wide characters, the return value is zero.

— Function: int strncasecmp (const char *s1, const char *s2, size_t n)

This function is like strncmp, except that differences in case are ignored. Like strcasecmp, it is locale dependent how uppercase and lowercase characters are related.

strncasecmp is a GNU extension.

— Function: int wcsncasecmp (const wchar_t *ws1, const wchar_t *s2, size_t n)

This function is like wcsncmp, except that differences in case are ignored. Like wcscasecmp, it is locale dependent how uppercase and lowercase characters are related.

wcsncasecmp is a GNU extension.

Here are some examples showing the use of strcmp and strncmp (equivalent examples can be constructed for the wide character functions). These examples assume the use of the ASCII character set. (If some other character set—say, EBCDIC—is used instead, then the glyphs are associated with different numeric codes, and the return values and ordering may differ.) 

     strcmp ("hello", "hello")
         ⇒ 0    /* These two strings are the same. */
     strcmp ("hello", "Hello")
         ⇒ 32   /* Comparisons are case-sensitive. */
     strcmp ("hello", "world")
         ⇒ -15  /* The character 'h' comes before 'w'. */
     strcmp ("hello", "hello, world")
         ⇒ -44  /* Comparing a null character against a comma. */
     strncmp ("hello", "hello, world", 5)
         ⇒ 0    /* The initial 5 characters are the same. */
     strncmp ("hello, world", "hello, stupid world!!!", 5)
         ⇒ 0    /* The initial 5 characters are the same. */
— Function: int strverscmp (const char *s1, const char *s2)

The strverscmp function compares the string s1 against s2, considering them as holding indices/version numbers. Return value follows the same conventions as found in the strverscmp function. In fact, if s1 and s2 contain no digits, strverscmp behaves like strcmp.

Basically, we compare strings normally (character by character), until we find a digit in each string - then we enter a special comparison mode, where each sequence of digits is taken as a whole. If we reach the end of these two parts without noticing a difference, we return to the standard comparison mode. There are two types of numeric parts: "integral" and "fractional" (those begin with a '0'). The types of the numeric parts affect the way we sort them: 

          strverscmp ("no digit", "no digit")
              ⇒ 0    /* same behavior as strcmp. */
          strverscmp ("item#99", "item#100")
              ⇒ <0   /* same prefix, but 99 < 100. */
          strverscmp ("alpha1", "alpha001")
              ⇒ >0   /* fractional part inferior to integral one. */
          strverscmp ("part1_f012", "part1_f01")
              ⇒ >0   /* two fractional parts. */
          strverscmp ("foo.009", "foo.0")
              ⇒ <0   /* idem, but with leading zeroes only. */

This function is especially useful when dealing with filename sorting, because filenames frequently hold indices/version numbers.

strverscmp is a GNU extension.

— Function: int bcmp (const void *a1, const void *a2, size_t size)

This is an obsolete alias for memcmp, derived from BSD.

Next: , Previous: String/Array Comparison, Up: String and Array Utilities

5.6 Collation Functions

In some locales, the conventions for lexicographic ordering differ from the strict numeric ordering of character codes. For example, in Spanish most glyphs with diacritical marks such as accents are not considered distinct letters for the purposes of collation. On the other hand, the two-character sequence ‘ll’ is treated as a single letter that is collated immediately after ‘l’.

You can use the functions strcoll and strxfrm (declared in the headers file string.h) and wcscoll and wcsxfrm (declared in the headers file wchar) to compare strings using a collation ordering appropriate for the current locale. The locale used by these functions in particular can be specified by setting the locale for the LC_COLLATE category; see Locales. In the standard C locale, the collation sequence for strcoll is the same as that for strcmp. Similarly, wcscoll and wcscmp are the same in this situation.

Effectively, the way these functions work is by applying a mapping to transform the characters in a string to a byte sequence that represents the string's position in the collating sequence of the current locale. Comparing two such byte sequences in a simple fashion is equivalent to comparing the strings with the locale's collating sequence.

The functions strcoll and wcscoll perform this translation implicitly, in order to do one comparison. By contrast, strxfrm and wcsxfrm perform the mapping explicitly. If you are making multiple comparisons using the same string or set of strings, it is likely to be more efficient to use strxfrm or wcsxfrm to transform all the strings just once, and subsequently compare