Table of Contents ***************** GNU M4 1 Introduction and preliminaries 1.1 Introduction to `m4' 1.2 Historical references 1.3 Problems and bugs 1.4 Using this manual 2 Invoking `m4' 2.1 Command line options for operation modes 2.2 Command line options for preprocessor features 2.3 Command line options for limits control 2.4 Command line options for frozen state 2.5 Command line options for debugging 2.6 Specifying input files on the command line 3 Lexical and syntactic conventions 3.1 Macro names 3.2 Quoting input to `m4' 3.3 Comments in `m4' input 3.4 Other kinds of input tokens 3.5 How `m4' copies input to output 4 How to invoke macros 4.1 Macro invocation 4.2 Preventing macro invocation 4.3 Macro arguments 4.4 On Quoting Arguments to macros 4.5 Macro expansion 5 How to define new macros 5.1 Defining a macro 5.2 Arguments to macros 5.3 Special arguments to macros 5.4 Deleting a macro 5.5 Renaming macros 5.6 Temporarily redefining macros 5.7 Indirect call of macros 5.8 Indirect call of builtins 6 Conditionals, loops, and recursion 6.1 Testing if a macro is defined 6.2 If-else construct, or multibranch 6.3 Recursion in `m4' 6.4 Iteration by counting 6.5 Iteration by list contents 7 How to debug macros and input 7.1 Displaying macro definitions 7.2 Tracing macro calls 7.3 Controlling debugging output 7.4 Saving debugging output 8 Input control 8.1 Deleting whitespace in input 8.2 Changing the quote characters 8.3 Changing the comment delimiters 8.4 Changing the lexical structure of words 8.5 Saving text until end of input 9 File inclusion 9.1 Including named files 9.2 Searching for include files 10 Diverting and undiverting output 10.1 Diverting output 10.2 Undiverting output 10.3 Diversion numbers 10.4 Discarding diverted text 11 Macros for text handling 11.1 Calculating length of strings 11.2 Searching for substrings 11.3 Searching for regular expressions 11.4 Extracting substrings 11.5 Translating characters 11.6 Substituting text by regular expression 11.7 Formatting strings (printf-like) 12 Macros for doing arithmetic 12.1 Decrement and increment operators 12.2 Evaluating integer expressions 13 Macros for running shell commands 13.1 Determining the platform 13.2 Executing simple commands 13.3 Reading the output of commands 13.4 Exit status 13.5 Making temporary files 14 Miscellaneous builtin macros 14.1 Printing error messages 14.2 Printing current location 14.3 Exiting from `m4' 15 Fast loading of frozen state 15.1 Using frozen files 15.2 Frozen file format 16 Compatibility with other versions of `m4' 16.1 Extensions in GNU M4 16.2 Facilities in System V `m4' not in GNU `m4' 16.3 Other incompatibilities 17 Correct version of some examples 17.1 Solution for `exch' 17.2 Solution for `forloop' 17.3 Solution for `foreach' 17.4 Solution for `m4wrap' 17.5 Solution for `cleardivert' 17.6 Solution for `capitalize' 17.7 Solution for `fatal_error' Appendix A How to make copies of the overall M4 package A.1 License for copying the M4 package Appendix B How to make copies of this manual B.1 License for copying this manual Appendix C Indices of concepts and macros C.1 Index for all `m4' macros C.2 Index for many concepts GNU M4 ****** This manual is for GNU M4 (version 1.4.11, 31 March 2008), a package containing an implementation of the m4 macro language. Copyright (C) 1989, 1990, 1991, 1992, 1993, 1994, 2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License." GNU `m4' is an implementation of the traditional UNIX macro processor. It is mostly SVR4 compatible, although it has some extensions (for example, handling more than 9 positional parameters to macros). `m4' also has builtin functions for including files, running shell commands, doing arithmetic, etc. Autoconf needs GNU `m4' for generating `configure' scripts, but not for running them. GNU `m4' was originally written by Rene' Seindal, with subsequent changes by Franc,ois Pinard and other volunteers on the Internet. All names and email addresses can be found in the files `m4-1.4.11/AUTHORS' and `m4-1.4.11/THANKS' from the GNU M4 distribution. This is release 1.4.11. It is now considered stable: future releases in the 1.4.x series are only meant to fix bugs, increase speed, or improve documentation. However... An experimental feature, which would improve `m4' usefulness, allows for changing the syntax for what is a "word" in `m4'. You should use: ./configure --enable-changeword if you want this feature compiled in. The current implementation slows down `m4' considerably and is hardly acceptable. In the future, `m4' 2.0 will come with a different set of new features that provide similar capabilities, but without the inefficiencies, so changeword will go away and _you should not count on it_. 1 Introduction and preliminaries ******************************** This first chapter explains what GNU `m4' is, where `m4' comes from, how to read and use this documentation, how to call the `m4' program, and how to report bugs about it. It concludes by giving tips for reading the remainder of the manual. The following chapters then detail all the features of the `m4' language. 1.1 Introduction to `m4' ======================== `m4' is a macro processor, in the sense that it copies its input to the output, expanding macros as it goes. Macros are either builtin or user-defined, and can take any number of arguments. Besides just doing macro expansion, `m4' has builtin functions for including named files, running shell commands, doing integer arithmetic, manipulating text in various ways, performing recursion, etc.... `m4' can be used either as a front-end to a compiler, or as a macro processor in its own right. The `m4' macro processor is widely available on all UNIXes, and has been standardized by POSIX. Usually, only a small percentage of users are aware of its existence. However, those who find it often become committed users. The popularity of GNU Autoconf, which requires GNU `m4' for _generating_ `configure' scripts, is an incentive for many to install it, while these people will not themselves program in `m4'. GNU `m4' is mostly compatible with the System V, Release 3 version, except for some minor differences. *Note Compatibility::, for more details. Some people find `m4' to be fairly addictive. They first use `m4' for simple problems, then take bigger and bigger challenges, learning how to write complex sets of `m4' macros along the way. Once really addicted, users pursue writing of sophisticated `m4' applications even to solve simple problems, devoting more time debugging their `m4' scripts than doing real work. Beware that `m4' may be dangerous for the health of compulsive programmers. 1.2 Historical references ========================= `GPM' was an important ancestor of `m4'. See C. Stratchey: "A General Purpose Macro generator", Computer Journal 8,3 (1965), pp. 225 ff. `GPM' is also succinctly described into David Gries classic "Compiler Construction for Digital Computers". The classic B. Kernighan and P.J. Plauger: "Software Tools", Addison-Wesley, Inc. (1976) describes and implements a Unix macro-processor language, which inspired Dennis Ritchie to write `m3', a macro processor for the AP-3 minicomputer. Kernighan and Ritchie then joined forces to develop the original `m4', as described in "The M4 Macro Processor", Bell Laboratories (1977). It had only 21 builtin macros. While `GPM' was more _pure_, `m4' is meant to deal with the true intricacies of real life: macros can be recognized without being pre-announced, skipping whitespace or end-of-lines is easier, more constructs are builtin instead of derived, etc. Originally, the Kernighan and Plauger macro-processor, and then `m3', formed the engine for the Rational FORTRAN preprocessor, that is, the `Ratfor' equivalent of `cpp'. Later, `m4' was used as a front-end for `Ratfor', `C' and `Cobol'. Rene' Seindal released his implementation of `m4', GNU `m4', in 1990, with the aim of removing the artificial limitations in many of the traditional `m4' implementations, such as maximum line length, macro size, or number of macros. The late Professor A. Dain Samples described and implemented a further evolution in the form of `M5': "User's Guide to the M5 Macro Language: 2nd edition", Electronic Announcement on comp.compilers newsgroup (1992). Franc,ois Pinard took over maintenance of GNU `m4' in 1992, until 1994 when he released GNU `m4' 1.4, which was the stable release for 10 years. It was at this time that GNU Autoconf decided to require GNU `m4' as its underlying engine, since all other implementations of `m4' had too many limitations. More recently, in 2004, Paul Eggert released 1.4.1 and 1.4.2 which addressed some long standing bugs in the venerable 1.4 release. Then in 2005, Gary V. Vaughan collected together the many patches to GNU `m4' 1.4 that were floating around the net and released 1.4.3 and 1.4.4. And in 2006, Eric Blake joined the team and prepared patches for the release of 1.4.5, 1.4.6, 1.4.7, and 1.4.8. The 1.4.x series remains open for bug fixes, including releases 1.4.9, 1.4.10, and 1.4.11 in 2007. Meanwhile, development has continued on new features for `m4', such as dynamic module loading and additional builtins. When complete, GNU `m4' 2.0 will start a new series of releases. 1.3 Problems and bugs ===================== If you have problems with GNU M4 or think you've found a bug, please report it. Before reporting a bug, make sure you've actually found a real bug. Carefully reread the documentation and see if it really says you can do what you're trying to do. If it's not clear whether you should be able to do something or not, report that too; it's a bug in the documentation! Before reporting a bug or trying to fix it yourself, try to isolate it to the smallest possible input file that reproduces the problem. Then send us the input file and the exact results `m4' gave you. Also say what you expected to occur; this will help us decide whether the problem was really in the documentation. Once you've got a precise problem, send e-mail to . Please include the version number of `m4' you are using. You can get this information with the command `m4 --version'. Also provide details about the platform you are executing on. Non-bug suggestions are always welcome as well. If you have questions about things that are unclear in the documentation or are just obscure features, please report them too. 1.4 Using this manual ===================== This manual contains a number of examples of `m4' input and output, and a simple notation is used to distinguish input, output and error messages from `m4'. Examples are set out from the normal text, and shown in a fixed width font, like this This is an example of an example! To distinguish input from output, all output from `m4' is prefixed by the string `=>', and all error messages by the string `error-->'. When showing how command line options affect matters, the command line is shown with a prompt `$ like this', otherwise, you can assume that a simple `m4' invocation will work. Thus: $ command line to invoke m4 Example of input line =>Output line from m4 error-->and an error message The sequence `^D' in an example indicates the end of the input file. The sequence `' refers to the newline character. The majority of these examples are self-contained, and you can run them with similar results by invoking `m4 -d'. In fact, the testsuite that is bundled in the GNU M4 package consists of the examples in this document! Some of the examples assume that your current directory is located where you unpacked the installation, so if you plan on following along, you may find it helpful to do this now: $ cd m4-1.4.11 As each of the predefined macros in `m4' is described, a prototype call of the macro will be shown, giving descriptive names to the arguments, e.g., -- Composite: example (STRING, [COUNT = `1'], [ARGUMENT]...) This is a sample prototype. There is not really a macro named `example', but this documents that if there were, it would be a Composite macro, rather than a Builtin. It requires at least one argument, STRING. Remember that in `m4', there must not be a space between the macro name and the opening parenthesis, unless it was intended to call the macro without any arguments. The brackets around COUNT and ARGUMENT show that these arguments are optional. If COUNT is omitted, the macro behaves as if count were `1', whereas if ARGUMENT is omitted, the macro behaves as if it were the empty string. A blank argument is not the same as an omitted argument. For example, `example(`a')', `example(`a',`1')', and `example(`a',`1',)' would behave identically with COUNT set to `1'; while `example(`a',)' and `example(`a',`')' would explicitly pass the empty string for COUNT. The ellipses (`...') show that the macro processes additional arguments after ARGUMENT, rather than ignoring them. All macro arguments in `m4' are strings, but some are given special interpretation, e.g., as numbers, file names, regular expressions, etc. The documentation for each macro will state how the parameters are interpreted, and what happens if the argument cannot be parsed according to the desired interpretation. Unless specified otherwise, a parameter specified to be a number is parsed as a decimal, even if the argument has leading zeros; and parsing the empty string as a number results in 0 rather than an error, although a warning will be issued. This document consistently writes and uses "builtin", without a hyphen, as if it were an English word. This is how the `builtin' primitive is spelled within `m4'. 2 Invoking `m4' *************** The format of the `m4' command is: `m4' [OPTION...] [FILE...] All options begin with `-', or if long option names are used, with `--'. A long option name need not be written completely, any unambiguous prefix is sufficient. POSIX requires `m4' to recognize arguments intermixed with files, even when `POSIXLY_CORRECT' is set in the environment. Most options take effect at startup regardless of their position, but some are documented below as taking effect after any files that occurred earlier in the command line. The argument `--' is a marker to denote the end of options. With short options, options that do not take arguments may be combined into a single command line argument with subsequent options, options with mandatory arguments may be provided either as a single command line argument or as two arguments, and options with optional arguments must be provided as a single argument. In other words, `m4 -QPDfoo -d a -d+f' is equivalent to `m4 -Q -P -D foo -d -d+f -- ./a', although the latter form is considered canonical. With long options, options with mandatory arguments may be provided with an equal sign (`=') in a single argument, or as two arguments, and options with optional arguments must be provided as a single argument. In other words, `m4 --def foo --debug a' is equivalent to `m4 --define=foo --debug= -- ./a', although the latter form is considered canonical (not to mention more robust, in case a future version of `m4' introduces an option named `--default'). `m4' understands the following options, grouped by functionality. 2.1 Command line options for operation modes ============================================ Several options control the overall operation of `m4': `--help' Print a help summary on standard output, then immediately exit `m4' without reading any input files or performing any other actions. `--version' Print the version number of the program on standard output, then immediately exit `m4' without reading any input files or performing any other actions. `-E' `--fatal-warnings' Controls the effect of warnings. If unspecified, then execution continues and exit status is unaffected when a warning is printed. If specified exactly once, warnings become fatal; when one is issued, execution continues, but the exit status will be non-zero. If specified multiple times, then execution halts with non-zero status the first time a warning is issued. The introduction of behavior levels is new to M4 1.4.9; for behavior consistent with earlier versions, you should specify `-E' twice. `-i' `--interactive' `-e' Makes this invocation of `m4' interactive. This means that all output will be unbuffered, and interrupts will be ignored. The spelling `-e' exists for compatibility with other `m4' implementations, and issues a warning because it may be withdrawn in a future version of GNU M4. `-P' `--prefix-builtins' Internally modify _all_ builtin macro names so they all start with the prefix `m4_'. For example, using this option, one should write `m4_define' instead of `define', and `m4___file__' instead of `__file__'. This option has no effect if `-R' is also specified. `-Q' `--quiet' `--silent' Suppress warnings, such as missing or superfluous arguments in macro calls, or treating the empty string as zero. `--warn-macro-sequence[=REGEXP]' Issue a warning if the regular expression REGEXP has a non-empty match in any macro definition (either by `define' or `pushdef'). Empty matches are ignored; therefore, supplying the empty string as REGEXP disables any warning. If the optional REGEXP is not supplied, then the default regular expression is `\$\({[^}]*}\|[0-9][0-9]+\)' (a literal `$' followed by multiple digits or by an open brace), since these sequences will change semantics in the default operation of GNU M4 2.0 (due to a change in how more than 9 arguments in a macro definition will be handled, *note Arguments::). Providing an alternate regular expression can provide a useful reverse lookup feature of finding where a macro is defined to have a given definition. `-W REGEXP' `--word-regexp=REGEXP' Use REGEXP as an alternative syntax for macro names. This experimental option will not be present in all GNU `m4' implementations (*note Changeword::). 2.2 Command line options for preprocessor features ================================================== Several options allow `m4' to behave more like a preprocessor. Macro definitions and deletions can be made on the command line, the search path can be altered, and the output file can track where the input came from. These features occur with the following options: `-D NAME[=VALUE]' `--define=NAME[=VALUE]' This enters NAME into the symbol table. If `=VALUE' is missing, the value is taken to be the empty string. The VALUE can be any string, and the macro can be defined to take arguments, just as if it was defined from within the input. This option may be given more than once; order with respect to file names is significant, and redefining the same NAME loses the previous value. `-I DIRECTORY' `--include=DIRECTORY' Make `m4' search DIRECTORY for included files that are not found in the current working directory. *Note Search Path::, for more details. This option may be given more than once. `-s' `--synclines' Generate synchronization lines, for use by the C preprocessor or other similar tools. Order is significant with respect to file names. This option is useful, for example, when `m4' is used as a front end to a compiler. Source file name and line number information is conveyed by directives of the form `#line LINENUM "FILE"', which are inserted as needed into the middle of the output. Such directives mean that the following line originated or was expanded from the contents of input file FILE at line LINENUM. The `"FILE"' part is often omitted when the file name did not change from the previous directive. Synchronization directives are always given on complete lines by themselves. When a synchronization discrepancy occurs in the middle of an output line, the associated synchronization directive is delayed until the next newline that does not occur in the middle of a quoted string or comment. define(`twoline', `1 2') =>#line 2 "stdin" => changecom(`/*', `*/') => define(`comment', `/*1 2*/') =>#line 5 => dnl no line hello =>#line 7 =>hello twoline =>1 =>#line 8 =>2 comment =>/*1 =>2*/ one comment `two three' =>#line 10 =>one /*1 =>2*/ two =>three goodbye =>#line 12 =>goodbye `-U NAME' `--undefine=NAME' This deletes any predefined meaning NAME might have. Obviously, only predefined macros can be deleted in this way. This option may be given more than once; undefining a NAME that does not have a definition is silently ignored. Order is significant with respect to file names. 2.3 Command line options for limits control =========================================== There are some limits within `m4' that can be tuned. For compatibility, `m4' also accepts some options that control limits in other implementations, but which are automatically unbounded (limited only by your hardware and operating system constraints) in GNU `m4'. `-G' `--traditional' Suppress all the extensions made in this implementation, compared to the System V version. *Note Compatibility::, for a list of these. `-H NUM' `--hashsize=NUM' Make the internal hash table for symbol lookup be NUM entries big. For better performance, the number should be prime, but this is not checked. The default is 509 entries. It should not be necessary to increase this value, unless you define an excessive number of macros. `-L NUM' `--nesting-limit=NUM' Artificially limit the nesting of macro calls to NUM levels, stopping program execution if this limit is ever exceeded. When not specified, nesting is limited to 1024 levels. A value of zero means unlimited; but then heavily nested code could potentially cause a stack overflow. The precise effect of this option might be more correctly associated with textual nesting than dynamic recursion. It has been useful when some complex `m4' input was generated by mechanical means. Most users would never need this option. If shown to be obtrusive, this option (which is still experimental) might well disappear. This option does _not_ have the ability to break endless rescanning loops, since these do not necessarily consume much memory or stack space. Through clever usage of rescanning loops, one can request complex, time-consuming computations from `m4' with useful results. Putting limitations in this area would break `m4' power. There are many pathological cases: `define(`a', `a')a' is only the simplest example (but *note Compatibility::). Expecting GNU `m4' to detect these would be a little like expecting a compiler system to detect and diagnose endless loops: it is a quite _hard_ problem in general, if not undecidable! `-B NUM' `-S NUM' `-T NUM' These options are present for compatibility with System V `m4', but do nothing in this implementation. They may disappear in future releases, and issue a warning to that effect. `-N NUM' `--diversions=NUM' These options are present only for compatibility with previous versions of GNU `m4', and were controlling the number of possible diversions which could be used at the same time. They do nothing, because there is no fixed limit anymore. They may disappear in future releases, and issue a warning to that effect. 2.4 Command line options for frozen state ========================================= GNU `m4' comes with a feature of freezing internal state (*note Frozen files::). This can be used to speed up `m4' execution when reusing a common initialization script. `-F FILE' `--freeze-state=FILE' Once execution is finished, write out the frozen state on the specified FILE. It is conventional, but not required, for FILE to end in `.m4f'. `-R FILE' `--reload-state=FILE' Before execution starts, recover the internal state from the specified frozen FILE. The options `-D', `-U', and `-t' take effect after state is reloaded, but before the input files are read. 2.5 Command line options for debugging ====================================== Finally, there are several options for aiding in debugging `m4' scripts. `-d[FLAGS]' `--debug[=FLAGS]' Set the debug-level according to the flags FLAGS. The debug-level controls the format and amount of information presented by the debugging functions. *Note Debug Levels::, for more details on the format and meaning of FLAGS. If omitted, FLAGS defaults to `aeq'. `--debugfile=FILE' `-o FILE' `--error-output=FILE' Redirect `dumpdef' output, debug messages, and trace output to the named FILE. Warnings, error messages, and `errprint' output are still printed to standard error. If unspecified, debug output goes to standard error; if empty, debug output is discarded. *Note Debug Output::, for more details. The spellings `-o' and `--error-output' are misleading and inconsistent with other GNU tools; for now they are silently accepted as synonyms of `--debugfile', but in a future version of M4, using them will cause a warning to be issued. `-l NUM' `--arglength=NUM' Restrict the size of the output generated by macro tracing to NUM characters per trace line. If unspecified or zero, output is unlimited. *Note Debug Levels::, for more details. `-t NAME' `--trace=NAME' This enables tracing for the macro NAME, at any point where it is defined. NAME need not be defined when this option is given. This option may be given more than once, and order is significant with respect to file names. *Note Trace::, for more details. 2.6 Specifying input files on the command line ============================================== The remaining arguments on the command line are taken to be input file names. If no names are present, standard input is read. A file name of `-' is taken to mean standard input. It is conventional, but not required, for input files to end in `.m4'. The input files are read in the sequence given. Standard input can be read more than once, so the file name `-' may appear multiple times on the command line; this makes a difference when input is from a terminal or other special file type. It is an error if an input file ends in the middle of argument collection, a comment, or a quoted string. The options `--define' (`-D'), `--undefine' (`-U'), `--synclines' (`-s'), and `--trace' (`-t') only take effect after processing input from any file names that occur earlier on the command line. For example, assume the file `foo' contains: $ cat foo bar The text `bar' can then be redefined over multiple uses of `foo': $ m4 -Dbar=hello foo -Dbar=world foo =>hello =>world If none of the input files invoked `m4exit' (*note M4exit::), the exit status of `m4' will be 0 for success, 1 for general failure (such as problems with reading an input file), and 63 for version mismatch (*note Using frozen files::). If you need to read a file whose name starts with a `-', you can specify it as `./-file', or use `--' to mark the end of options. 3 Lexical and syntactic conventions *********************************** As `m4' reads its input, it separates it into "tokens". A token is either a name, a quoted string, or any single character, that is not a part of either a name or a string. Input to `m4' can also contain comments. GNU `m4' does not yet understand multibyte locales; all operations are byte-oriented rather than character-oriented (although if your locale uses a single byte encoding, such as ISO-8859-1, you will not notice a difference). However, `m4' is eight-bit clean, so you can use non-ASCII characters in quoted strings (*note Changequote::), comments (*note Changecom::), and macro names (*note Indir::), with the exception of the NUL character (the zero byte `'\0''). 3.1 Macro names =============== A name is any sequence of letters, digits, and the character `_' (underscore), where the first character is not a digit. `m4' will use the longest such sequence found in the input. If a name has a macro definition, it will be subject to macro expansion (*note Macros::). Names are case-sensitive. Examples of legal names are: `foo', `_tmp', and `name01'. 3.2 Quoting input to `m4' ========================= A quoted string is a sequence of characters surrounded by quote strings, defaulting to ``' and `'', where the nested begin and end quotes within the string are balanced. The value of a string token is the text, with one level of quotes stripped off. Thus `' => is the empty string, and double-quoting turns into single-quoting. ``quoted'' =>`quoted' The quote characters can be changed at any time, using the builtin macro `changequote'. *Note Changequote::, for more information. 3.3 Comments in `m4' input ========================== Comments in `m4' are normally delimited by the characters `#' and newline. All characters between the comment delimiters are ignored, but the entire comment (including the delimiters) is passed through to the output--comments are _not_ discarded by `m4'. Comments cannot be nested, so the first newline after a `#' ends the comment. The commenting effect of the begin-comment string can be inhibited by quoting it. $ m4 `quoted text' # `commented text' =>quoted text # `commented text' `quoting inhibits' `#' `comments' =>quoting inhibits # comments The comment delimiters can be changed to any string at any time, using the builtin macro `changecom'. *Note Changecom::, for more information. 3.4 Other kinds of input tokens =============================== Any character, that is neither a part of a name, nor of a quoted string, nor a comment, is a token by itself. When not in the context of macro expansion, all of these tokens are just copied to output. However, during macro expansion, whitespace characters (space, tab, newline, formfeed, carriage return, vertical tab), parentheses (`(' and `)'), comma (`,'), and dollar (`$') have additional roles, explained later. 3.5 How `m4' copies input to output =================================== As `m4' reads the input token by token, it will copy each token directly to the output immediately. The exception is when it finds a word with a macro definition. In that case `m4' will calculate the macro's expansion, possibly reading more input to get the arguments. It then inserts the expansion in front of the remaining input. In other words, the resulting text from a macro call will be read and parsed into tokens again. `m4' expands a macro as soon as possible. If it finds a macro call when collecting the arguments to another, it will expand the second call first. This process continues until there are no more macro calls to expand and all the input has been consumed. For a running example, examine how `m4' handles this input: format(`Result is %d', eval(`2**15')) First, `m4' sees that the token `format' is a macro name, so it collects the tokens `(', ``Result is %d'', `,', and ` ', before encountering another potential macro. Sure enough, `eval' is a macro name, so the nested argument collection picks up `(', ``2**15'', and `)', invoking the eval macro with the lone argument of `2**15'. The expansion of `eval(2**15)' is `32768', which is then rescanned as the five tokens `3', `2', `7', `6', and `8'; and combined with the next `)', the format macro now has all its arguments, as if the user had typed: format(`Result is %d', 32768) The format macro expands to `Result is 32768', and we have another round of scanning for the tokens `Result', ` ', `is', ` ', `3', `2', `7', `6', and `8'. None of these are macros, so the final output is =>Result is 32768 As a more complicated example, we will contrast an actual code example from the Gnulib project(1), showing both a buggy approach and the desired results. The user desires to output a shell assignment statement that takes its argument and turns it into a shell variable by converting it to uppercase and prepending a prefix. The original attempt looks like this: changequote([,])dnl define([gl_STRING_MODULE_INDICATOR], [ dnl comment GNULIB_]translit([$1],[a-z],[A-Z])[=1 ])dnl gl_STRING_MODULE_INDICATOR([strcase]) => => GNULIB_strcase=1 => Oops - the argument did not get capitalized. And although the manual is not able to easily show it, both lines that appear empty actually contain two trailing spaces. By stepping through the parse, it is easy to see what happened. First, `m4' sees the token `changequote', which it recognizes as a macro, followed by `(', `[', `,', `]', and `)' to form the argument list. The macro expands to the empty string, but changes the quoting characters to something more useful for generating shell code (unbalanced ``' and `'' appear all the time in shell scripts, but unbalanced `[]' tend to be rare). Also in the first line, `m4' sees the token `dnl', which it recognizes as a builtin macro that consumes the rest of the line, resulting in no output for that line. The second line starts a macro definition. `m4' sees the token `define', which it recognizes as a macro, followed by a `(', `[gl_STRING_MODULE_INDICATOR]', and `,'. Because an unquoted comma was encountered, the first argument is known to be the expansion of the single-quoted string token, or `gl_STRING_MODULE_INDICATOR'. Next, `m4' sees `', ` ', and ` ', but this whitespace is discarded as part of argument collection. Then comes a rather lengthy single-quoted string token, `[ dnl comment GNULIB_]'. This is followed by the token `translit', which `m4' recognizes as a macro name, so a nested macro expansion has started. The arguments to the `translit' are found by the tokens `(', `[$1]', `,', `[a-z]', `,', `[A-Z]', and finally `)'. All three string arguments are expanded (or in other words, the quotes are stripped), and since neither `$' nor `1' need capitalization, the result of the macro is `$1'. This expansion is rescanned, resulting in the two literal characters `$' and `1'. Scanning of the outer macro resumes, and picks up with `[=1 ]', and finally `)'. The collected pieces of expanded text are concatenated, with the end result that the macro `gl_STRING_MODULE_INDICATOR' is now defined to be the sequence ` dnl comment GNULIB_$1=1 '. Once again, `dnl' is recognized and avoids a newline in the output. The final line is then parsed, beginning with ` ' and ` ' that are output literally. Then `gl_STRING_MODULE_INDICATOR' is recognized as a macro name, with an argument list of `(', `[strcase]', and `)'. Since the definition of the macro contains the sequence `$1', that sequence is replaced with the argument `strcase' prior to starting the rescan. The rescan sees `' and four spaces, which are output literally, then `dnl', which discards the text ` comment'. Next comes four more spaces, also output literally, and the token `GNULIB_strcase', which resulted from the earlier parameter substitution. Since that is not a macro name, it is output literally, followed by the literal tokens `=', `1', `', and two more spaces. Finally, the original `' seen after the macro invocation is scanned and output literally. Now for a corrected approach. This rearranges the use of newlines and whitespace so that less whitespace is output (which, although harmless to shell scripts, can be visually unappealing), and fixes the quoting issues so that the capitalization occurs when the macro `gl_STRING_MODULE_INDICATOR' is invoked, rather then when it is defined. changequote([,])dnl define([gl_STRING_MODULE_INDICATOR], [dnl comment GNULIB_[]translit([$1], [a-z], [A-Z])=1dnl ])dnl gl_STRING_MODULE_INDICATOR([strcase]) => GNULIB_STRCASE=1 The parsing of the first line is unchanged. The second line sees the name of the macro to define, then sees the discarded `' and two spaces, as before. But this time, the next token is `[dnl comment GNULIB_[]translit([$1], [a-z], [A-Z])=1dnl]', which includes nested quotes, followed by `)' to end the macro definition and `dnl' to skip the newline. No early expansion of `translit' occurs, so the entire string becomes the definition of the macro. The final line is then parsed, beginning with two spaces that are output literally, and an invocation of `gl_STRING_MODULE_INDICATOR' with the argument `strcase'. Again, the `$1' in the macro definition is substituted prior to rescanning. Rescanning first encounters `dnl', and discards ` comment'. Then two spaces are output literally. Next comes the token `GNULIB_', but that is not a macro, so it is output literally. The token `[]' is an empty string, so it does not affect output. Then the token `translit' is encountered. This time, the arguments to `translit' are parsed as `(', `[strcase]', `,', ` ', `[a-z]', `,', ` ', `[A-Z]', and `)'. The two spaces are discarded, and the translit results in the desired result `STRCASE'. This is rescanned, but since it is not a macro name, it is output literally. Then the scanner sees `=' and `1', which are output literally, followed by `dnl' which discards the rest of the definition of `gl_STRING_MODULE_INDICATOR'. The newline at the end of output is the literal `' that appeared after the invocation of the macro. The order in which `m4' expands the macros can be further explored using the trace facilities of GNU `m4' (*note Trace::). ---------- Footnotes ---------- (1) Derived from a patch in `http://lists.gnu.org/archive/html/bug-gnulib/2007-01/msg00389.html', and a followup patch in `http://lists.gnu.org/archive/html/bug-gnulib/2007-02/msg00000.html' 4 How to invoke macros ********************** This chapter covers macro invocation, macro arguments and how macro expansion is treated. 4.1 Macro invocation ==================== Macro invocations has one of the forms name which is a macro invocation without any arguments, or name(arg1, arg2, ..., argN) which is a macro invocation with N arguments. Macros can have any number of arguments. All arguments are strings, but different macros might interpret the arguments in different ways. The opening parenthesis _must_ follow the NAME directly, with no spaces in between. If it does not, the macro is called with no arguments at all. For a macro call to have no arguments, the parentheses _must_ be left out. The macro call name() is a macro call with one argument, which is the empty string, not a call with no arguments. 4.2 Preventing macro invocation =============================== An innovation of the `m4' language, compared to some of its predecessors (like Stratchey's `GPM', for example), is the ability to recognize macro calls without resorting to any special, prefixed invocation character. While generally useful, this feature might sometimes be the source of spurious, unwanted macro calls. So, GNU `m4' offers several mechanisms or techniques for inhibiting the recognition of names as macro calls. First of all, many builtin macros cannot meaningfully be called without arguments. As a GNU extension, for any of these macros, whenever an opening parenthesis does not immediately follow their name, the builtin macro call is not triggered. This solves the most usual cases, like for `include' or `eval'. Later in this document, the sentence "This macro is recognized only with parameters" refers to this specific provision of GNU M4, also known as a blind builtin macro. For the builtins defined by POSIX that bear this disclaimer, POSIX specifically states that invoking those builtins without arguments is unspecified, because many other implementations simply invoke the builtin as though it were given one empty argument instead. $ m4 eval =>eval eval(`1') =>1 There is also a command line option (`--prefix-builtins', or `-P', *note Invoking m4: Operation modes.) that renames all builtin macros with a prefix of `m4_' at startup. The option has no effect whatsoever on user defined macros. For example, with this option, one has to write `m4_dnl' and even `m4_m4exit'. It also has no effect on whether a macro requires parameters. $ m4 -P eval =>eval eval(`1') =>eval(1) m4_eval =>m4_eval m4_eval(`1') =>1 Another alternative is to redefine problematic macros to a name less likely to cause conflicts, *Note Definitions::. If your version of GNU `m4' has the `changeword' feature compiled in, it offers far more flexibility in specifying the syntax of macro names, both builtin or user-defined. *Note Changeword::, for more information on this experimental feature. Of course, the simplest way to prevent a name from being interpreted as a call to an existing macro is to quote it. The remainder of this section studies a little more deeply how quoting affects macro invocation, and how quoting can be used to inhibit macro invocation. Even if quoting is usually done over the whole macro name, it can also be done over only a few characters of this name (provided, of course, that the unquoted portions are not also a macro). It is also possible to quote the empty string, but this works only _inside_ the name. For example: `divert' =>divert `d'ivert =>divert di`ver't =>divert div`'ert =>divert all yield the string `divert'. While in both: `'divert => divert`' => the `divert' builtin macro will be called, which expands to the empty string. The output of macro evaluations is always rescanned. In the following example, the input `x`'y' yields the string `bCD', exactly as if `m4' has been given `substr(ab`'cde, `1', `3')' as input: define(`cde', `CDE') => define(`x', `substr(ab') => define(`y', `cde, `1', `3')') => x`'y =>bCD Unquoted strings on either side of a quoted string are subject to being recognized as macro names. In the following example, quoting the empty string allows for the second `macro' to be recognized as such: define(`macro', `m') => macro(`m')macro =>mmacro macro(`m')`'macro =>mm Quoting may prevent recognizing as a macro name the concatenation of a macro expansion with the surrounding characters. In this example: define(`macro', `di$1') => macro(`v')`ert' =>divert macro(`v')ert => the input will produce the string `divert'. When the quotes were removed, the `divert' builtin was called instead. 4.3 Macro arguments =================== When a name is seen, and it has a macro definition, it will be expanded as a macro. If the name is followed by an opening parenthesis, the arguments will be collected before the macro is called. If too few arguments are supplied, the missing arguments are taken to be the empty string. However, some builtins are documented to behave differently for a missing optional argument than for an explicit empty string. If there are too many arguments, the excess arguments are ignored. Unquoted leading whitespace is stripped off all arguments, but whitespace generated by a macro expansion or occurring after a macro that expanded to an empty string remains intact. Whitespace includes space, tab, newline, carriage return, vertical tab, and formfeed. define(`macro', `$1') => macro( unquoted leading space lost) =>unquoted leading space lost macro(` quoted leading space kept') => quoted leading space kept macro( divert `unquoted space kept after expansion') => unquoted space kept after expansion macro(macro(` ')`whitespace from expansion kept') => =>whitespace from expansion kept macro(`unquoted trailing whitespace kept' ) =>unquoted trailing whitespace kept => Normally `m4' will issue warnings if a builtin macro is called with an inappropriate number of arguments, but it can be suppressed with the `--quiet' command line option (or `--silent', or `-Q', *note Invoking m4: Operation modes.). For user defined macros, there is no check of the number of arguments given. $ m4 index(`abc') error-->m4:stdin:1: Warning: too few arguments to builtin `index' =>0 index(`abc',) =>0 index(`abc', `b', `ignored') error-->m4:stdin:3: Warning: excess arguments to builtin `index' ignored =>1 $ m4 -Q index(`abc') =>0 index(`abc',) =>0 index(`abc', `b', `ignored') =>1 Macros are expanded normally during argument collection, and whatever commas, quotes and parentheses that might show up in the resulting expanded text will serve to define the arguments as well. Thus, if FOO expands to `, b, c', the macro call bar(a foo, d) is a macro call with four arguments, which are `a ', `b', `c' and `d'. To understand why the first argument contains whitespace, remember that unquoted leading whitespace is never part of an argument, but trailing whitespace always is. It is possible for a macro's definition to change during argument collection, in which case the expansion uses the definition that was in effect at the time the opening `(' was seen. define(`f', `1') => f(define(`f', `2')) =>1 f =>2 It is an error if the end of file occurs while collecting arguments. hello world =>hello world define( ^D error-->m4:stdin:2: ERROR: end of file in argument list 4.4 On Quoting Arguments to macros ================================== Each argument has unquoted leading whitespace removed. Within each argument, all unquoted parentheses must match. For example, if FOO is a macro, foo(() (`(') `(') is a macro call, with one argument, whose value is `() (() ('. Commas separate arguments, except when they occur inside quotes, comments, or unquoted parentheses. *Note Pseudo Arguments::, for examples. It is common practice to quote all arguments to macros, unless you are sure you want the arguments expanded. Thus, in the above example with the parentheses, the `right' way to do it is like this: foo(`() (() (') It is, however, in certain cases necessary (because nested expansion must occur to create the arguments for the outer macro) or convenient (because it uses fewer characters) to leave out quotes for some arguments, and there is nothing wrong in doing it. It just makes life a bit harder, if you are not careful to follow a consistent quoting style. For consistency, this manual follows the rule of thumb that each layer of parentheses introduces another layer of single quoting, except when showing the consequences of quoting rules. This is done even when the quoted string cannot be a macro, such as with integers when you have not changed the syntax via `changeword' (*note Changeword::). The quoting rule of thumb of one level of quoting per parentheses has a nice property: when a macro name appears inside parentheses, you can determine when it will be expanded. If it is not quoted, it will be expanded prior to the outer macro, so that its expansion becomes the argument. If it is single-quoted, it will be expanded after the outer macro. And if it is double-quoted, it will be used as literal text instead of a macro name. define(`active', `ACT, IVE') => define(`show', `$1 $1') => show(active) =>ACT ACT show(`active') =>ACT, IVE ACT, IVE show(``active'') =>active active 4.5 Macro expansion =================== When the arguments, if any, to a macro call have been collected, the macro is expanded, and the expansion text is pushed back onto the input (unquoted), and reread. The expansion text from one macro call might therefore result in more macros being called, if the calls are included, completely or partially, in the first macro calls' expansion. Taking a very simple example, if FOO expands to `bar', and BAR expands to `Hello', the input $ m4 -Dbar=Hello -Dfoo=bar foo =>Hello will expand first to `bar', and when this is reread and expanded, into `Hello'. 5 How to define new macros ************************** Macros can be defined, redefined and deleted in several different ways. Also, it is possible to redefine a macro without losing a previous value, and bring back the original value at a later time. 5.1 Defining a macro ==================== The normal way to define or redefine macros is to use the builtin `define': -- Builtin: define (NAME, [EXPANSION]) Defines NAME to expand to EXPANSION. If EXPANSION is not given, it is taken to be empty. The expansion of `define' is void. The macro `define' is recognized only with parameters. The following example defines the macro FOO to expand to the text `Hello World.'. define(`foo', `Hello world.') => foo =>Hello world. The empty line in the output is there because the newline is not a part of the macro definition, and it is consequently copied to the output. This can be avoided by use of the macro `dnl'. *Note Dnl::, for details. The first argument to `define' should be quoted; otherwise, if the macro is already defined, you will be defining a different macro. This example shows the problems with underquoting, since we did not want to redefine `one': define(foo, one) => define(foo, two) => one =>two GNU `m4' normally replaces only the _topmost_ definition of a macro if it has several definitions from `pushdef' (*note Pushdef::). Some other implementations of `m4' replace all definitions of a macro with `define'. *Note Incompatibilities::, for more details. As a GNU extension, the first argument to `define' does not have to be a simple word. It can be any text string, even the empty string. A macro with a non-standard name cannot be invoked in the normal way, as the name is not recognized. It can only be referenced by the builtins `indir' (*note Indir::) and `defn' (*note Defn::). Arrays and associative arrays can be simulated by using non-standard macro names. -- Composite: array (INDEX) -- Composite: array_set (INDEX, [VALUE]) Provide access to entries within an array. `array' reads the entry at location INDEX, and `array_set' assigns VALUE to location INDEX. define(`array', `defn(format(``array[%d]'', `$1'))') => define(`array_set', `define(format(``array[%d]'', `$1'), `$2')') => array_set(`4', `array element no. 4') => array_set(`17', `array element no. 17') => array(`4') =>array element no. 4 array(eval(`10 + 7')) =>array element no. 17 Change the `%d' to `%s' and it is an associative array. 5.2 Arguments to macros ======================= Macros can have arguments. The Nth argument is denoted by `$n' in the expansion text, and is replaced by the Nth actual argument, when the macro is expanded. Replacement of arguments happens before rescanning, regardless of how many nesting levels of quoting appear in the expansion. Here is an example of a macro with two arguments. -- Composite: exch (ARG1, ARG2) Expands to ARG2 followed by ARG1, effectively exchanging their order. define(`exch', `$2, $1') => exch(`arg1', `arg2') =>arg2, arg1 This can be used, for example, if you like the arguments to `define' to be reversed. define(`exch', `$2, $1') => define(exch(``expansion text'', ``macro'')) => macro =>expansion text *Note Quoting Arguments::, for an explanation of the double quotes. (You should try and improve this example so that clients of `exch' do not have to double quote; or *note Answers: Improved exch.). As a special case, the zeroth argument, `$0', is always the name of the macro being expanded. define(`test', ``Macro name: $0'') => test =>Macro name: test If you want quoted text to appear as part of the expansion text, remember that quotes can be nested in quoted strings. Thus, in define(`foo', `This is macro `foo'.') => foo =>This is macro foo. The `foo' in the expansion text is _not_ expanded, since it is a quoted string, and not a name. GNU `m4' allows the number following the `$' to consist of one or more digits, allowing macros to have any number of arguments. The extension of accepting multiple digits is incompatible with POSIX, and is different than traditional implementations of `m4', which only recognize one digit. Therefore, future versions of GNU M4 will phase out this feature. To portably access beyond the ninth argument, you can use the `argn' macro documented later (*note Shift::). POSIX also states that `$' followed immediately by `{' in a macro definition is implementation-defined. This version of M4 passes the literal characters `${' through unchanged, but M4 2.0 will implement an optional feature similar to `sh', where `${11}' expands to the eleventh argument, to replace the current recognition of `$11'. Meanwhile, if you want to guarantee that you will get a literal `${' in output when expanding a macro, even when you upgrade to M4 2.0, you can use nested quoting to your advantage: define(`foo', `single quoted $`'{1} output') => define(`bar', ``double quoted $'`{2} output'') => foo(`a', `b') =>single quoted ${1} output bar(`a', `b') =>double quoted ${2} output To help you detect places in your M4 input files that might change in behavior due to the changed behavior of M4 2.0, you can use the `--warn-macro-sequence' command-line option (*note Invoking m4: Operation modes.) with the default regular expression. This will add a warning any time a macro definition includes `$' followed by multiple digits, or by `{'. The warning is not enabled by default, because it triggers a number of warnings in Autoconf 2.61 (and Autoconf uses `-E' to treat warnings as errors), and because it will still be possible to restore older behavior in M4 2.0. $ m4 --warn-macro-sequence define(`foo', `$001 ${1} $1') error-->m4:stdin:1: Warning: definition of `foo' contains sequence `$001' error-->m4:stdin:1: Warning: definition of `foo' contains sequence `${1}' => foo(`bar') =>bar ${1} bar 5.3 Special arguments to macros =============================== There is a special notation for the number of actual arguments supplied, and for all the actual arguments. The number of actual arguments in a macro call is denoted by `$#' in the expansion text. -- Composite: nargs (...) Expands to a count of the number of arguments supplied. define(`nargs', `$#') => nargs =>0 nargs() =>1 nargs(`arg1', `arg2', `arg3') =>3 nargs(`commas can be quoted, like this') =>1 nargs(arg1#inside comments, commas do not separate arguments still arg1) =>1 nargs((unquoted parentheses, like this, group arguments)) =>1 Remember that `#' defaults to the comment character; if you forget quotes to inhibit the comment behavior, your macro definition may not end where you expected. dnl Attempt to define a macro to just `$#' define(underquoted, $#) oops) => underquoted =>0) =>oops The notation `$*' can be used in the expansion text to denote all the actual arguments, unquoted, with commas in between. For example define(`echo', `$*') => echo(arg1, arg2, arg3 , arg4) =>arg1,arg2,arg3 ,arg4 Often each argument should be quoted, and the notation `$@' handles that. It is just like `$*', except that it quotes each argument. A simple example of that is: define(`echo', `$@') => echo(arg1, arg2, arg3 , arg4) =>arg1,arg2,arg3 ,arg4 Where did the quotes go? Of course, they were eaten, when the expanded text were reread by `m4'. To show the difference, try define(`echo1', `$*') => define(`echo2', `$@') => define(`foo', `This is macro `foo'.') => echo1(foo) =>This is macro This is macro foo.. echo1(`foo') =>This is macro foo. echo2(foo) =>This is macro foo. echo2(`foo') =>foo *Note Trace::, if you do not understand this. As another example of the difference, remember that comments encountered in arguments are passed untouched to the macro, and that quoting disables comments. define(`echo1', `$*') => define(`echo2', `$@') => define(`foo', `bar') => echo1(#foo'foo foo) =>#foo'foo =>bar echo2(#foo'foo foo) =>#foobar =>bar' A `$' sign in the expansion text, that is not followed by anything `m4' understands, is simply copied to the macro expansion, as any other text is. define(`foo', `$$$ hello $$$') => foo =>$$$ hello $$$ If you want a macro to expand to something like `$12', the judicious use of nested quoting can put a safe character between the `$' and the next character, relying on the rescanning to remove the nested quote. This will prevent `m4' from interpreting the `$' sign as a reference to an argument. define(`foo', `no nested quote: $1') => foo(`arg') =>no nested quote: arg define(`foo', `nested quote around $: `$'1') => foo(`arg') =>nested quote around $: $1 define(`foo', `nested empty quote after $: $`'1') => foo(`arg') =>nested empty quote after $: $1 define(`foo', `nested quote around next character: $`1'') => foo(`arg') =>nested quote around next character: $1 define(`foo', `nested quote around both: `$1'') => foo(`arg') =>nested quote around both: arg 5.4 Deleting a macro ==================== A macro definition can be removed with `undefine': -- Builtin: undefine (NAME...) For each argument, remove the macro NAME. The macro names must necessarily be quoted, since they will be expanded otherwise. The expansion of `undefine' is void. The macro `undefine' is recognized only with parameters. foo bar blah =>foo bar blah define(`foo', `some')define(`bar', `other')define(`blah', `text') => foo bar blah =>some other text undefine(`foo') => foo bar blah =>foo other text undefine(`bar', `blah') => foo bar blah =>foo bar blah Undefining a macro inside that macro's expansion is safe; the macro still expands to the definition that was in effect at the `('. define(`f', ``$0':$1') => f(f(f(undefine(`f')`hello world'))) =>f:f:f:hello world f(`bye') =>f(bye) It is not an error for NAME to have no macro definition. In that case, `undefine' does nothing. 5.5 Renaming macros =================== It is possible to rename an already defined macro. To do this, you need the builtin `defn': -- Builtin: defn (NAME...) Expands to the _quoted definition_ of each NAME. If an argument is not a defined macro, the expansion for that argument is empty. If NAME is a user-defined macro, the quoted definition is simply the quoted expansion text. If, instead, there is only one NAME and it is a builtin, the expansion is a special token, which points to the builtin's internal definition. This token is only meaningful as the second argument to `define' (and `pushdef'), and is silently converted to an empty string in most other contexts. Combining a builtin with anything else is not supported; a warning is issued and the builtin is omitted from the final expansion. The macro `defn' is recognized only with parameters. Its normal use is best understood through an example, which shows how to rename `undefine' to `zap': define(`zap', defn(`undefine')) => zap(`undefine') => undefine(`zap') =>undefine(zap) In this way, `defn' can be used to copy macro definitions, and also definitions of builtin macros. Even if the original macro is removed, the other name can still be used to access the definition. The fact that macro definitions can be transferred also explains why you should use `$0', rather than retyping a macro's name in its definition: define(`foo', `This is `$0'') => define(`bar', defn(`foo')) => bar =>This is bar Macros used as string variables should be referred through `defn', to avoid unwanted expansion of the text: define(`string', `The macro dnl is very useful ') => string =>The macro defn(`string') =>The macro dnl is very useful => However, it is important to remember that `m4' rescanning is purely textual. If an unbalanced end-quote string occurs in a macro definition, the rescan will see that embedded quote as the termination of the quoted string, and the remainder of the macro's definition will be rescanned unquoted. Thus it is a good idea to avoid unbalanced end-quotes in macro definitions or arguments to macros. define(`foo', a'a) => define(`a', `A') => define(`echo', `$@') => foo =>A'A defn(`foo') =>aA' echo(foo) =>AA' On the other hand, it is possible to exploit the fact that `defn' can concatenate multiple macros prior to the rescanning phase, in order to join the definitions of macros that, in isolation, have unbalanced quotes. This is particularly useful when one has used several macros to accumulate text that M4 should rescan as a whole. In the example below, note how the use of `defn' on `l' in isolation opens a string, which is not closed until the next line; but used on `l' and `r' together results in nested quoting. define(`l', `<[>')define(`r', `<]>') => changequote(`[', `]') => defn([l])defn([r]) ]) =><[>]defn([r]) =>) defn([l], [r]) =><[>][<]> Using `defn' to generate special tokens for builtin macros outside of expected contexts can sometimes trigger warnings. But most of the time, such tokens are silently converted to the empty string. $ m4 -d defn(`defn') => define(defn(`divnum'), `cannot redefine a builtin token') error-->m4:stdin:2: Warning: define: invalid macro name ignored => divnum =>0 len(defn(`divnum')) =>0 Also note that `defn' with multiple arguments can only join text macros, not builtins, although a future version of GNU M4 may lift this restriction. $ m4 -d define(`a', `A')define(`AA', `b') => traceon(`defn', `define') => defn(`a', `divnum', `a') error-->m4:stdin:3: Warning: cannot concatenate builtin `divnum' error-->m4trace: -1- defn(`a', `divnum', `a') -> ``A'`A'' =>AA define(`mydivnum', defn(`divnum', `divnum'))mydivnum error-->m4:stdin:4: Warning: cannot concatenate builtin `divnum' error-->m4:stdin:4: Warning: cannot concatenate builtin `divnum' error-->m4trace: -2- defn(`divnum', `divnum') error-->m4trace: -1- define(`mydivnum', `') => traceoff(`defn', `define') => 5.6 Temporarily redefining macros ================================= It is possible to redefine a macro temporarily, reverting to the previous definition at a later time. This is done with the builtins `pushdef' and `popdef': -- Builtin: pushdef (NAME, [EXPANSION]) -- Builtin: popdef (NAME...) Analogous to `define' and `undefine'. These macros work in a stack-like fashion. A macro is temporarily redefined with `pushdef', which replaces an existing definition of NAME, while saving the previous definition, before the new one is installed. If there is no previous definition, `pushdef' behaves exactly like `define'. If a macro has several definitions (of which only one is accessible), the topmost definition can be removed with `popdef'. If there is no previous definition, `popdef' behaves like `undefine'. The expansion of both `pushdef' and `popdef' is void. The macros `pushdef' and `popdef' are recognized only with parameters. define(`foo', `Expansion one.') => foo =>Expansion one. pushdef(`foo', `Expansion two.') => foo =>Expansion two. pushdef(`foo', `Expansion three.') => pushdef(`foo', `Expansion four.') => popdef(`foo') => foo =>Expansion three. popdef(`foo', `foo') => foo =>Expansion one. popdef(`foo') => foo =>foo If a macro with several definitions is redefined with `define', the topmost definition is _replaced_ with the new definition. If it is removed with `undefine', _all_ the definitions are removed, and not only the topmost one. However, POSIX allows other implementations that treat `define' as replacing an entire stack of definitions with a single new definition, so to be portable to other implementations, it may be worth explicitly using `popdef' and `pushdef' rather than relying on the GNU behavior of `define'. define(`foo', `Expansion one.') => foo =>Expansion one. pushdef(`foo', `Expansion two.') => foo =>Expansion two. define(`foo', `Second expansion two.') => foo =>Second expansion two. undefine(`foo') => foo =>foo Local variables within macros are made with `pushdef' and `popdef'. At the start of the macro a new definition is pushed, within the macro it is manipulated and at the end it is popped, revealing the former definition. It is possible to temporarily redefine a builtin with `pushdef' and `defn'. 5.7 Indirect call of macros =========================== Any macro can be called indirectly with `indir': -- Builtin: indir (NAME, [ARGS...]) Results in a call to the macro NAME, which is passed the rest of the arguments ARGS. If NAME is not defined, an error message is printed, and the expansion is void. The macro `indir' is recognized only with parameters. This can be used to call macros with computed or "invalid" names (`define' allows such names to be defined): define(`$$internal$macro', `Internal macro (name `$0')') => $$internal$macro =>$$internal$macro indir(`$$internal$macro') =>Internal macro (name $$internal$macro) The point is, here, that larger macro packages can have private macros defined, that will not be called by accident. They can _only_ be called through the builtin `indir'. One other point to observe is that argument collection occurs before `indir' invokes NAME, so if argument collection changes the value of NAME, that will be reflected in the final expansion. This is different than the behavior when invoking macros directly, where the definition that was in effect before argument collection is used. $ m4 -d define(`f', `1') => f(define(`f', `2')) =>1 indir(`f', define(`f', `3')) =>3 indir(`f', undefine(`f')) error-->m4:stdin:4: undefined macro `f' => When handed the result of `defn' (*note Defn::) as one of its arguments, `indir' defers to the invoked NAME for whether a token representing a builtin is recognized or flattened to the empty string. $ m4 -d indir(defn(`defn'), `divnum') error-->m4:stdin:1: Warning: indir: invalid macro name ignored => indir(`define', defn(`defn'), `divnum') error-->m4:stdin:2: Warning: define: invalid macro name ignored => indir(`define', `foo', defn(`divnum')) => foo =>0 indir(`divert', defn(`foo')) error-->m4:stdin:5: empty string treated as 0 in builtin `divert' => 5.8 Indirect call of builtins ============================= Builtin macros can be called indirectly with `builtin': -- Builtin: builtin (NAME, [ARGS...]) Results in a call to the builtin NAME, which is passed the rest of the arguments ARGS. If NAME does not name a builtin, an error message is printed, and the expansion is void. The macro `builtin' is recognized only with parameters. This can be used even if NAME has been given another definition that has covered the original, or been undefined so that no macro maps to the builtin. pushdef(`define', `hidden') => undefine(`undefine') => define(`foo', `bar') =>hidden foo =>foo builtin(`define', `foo', defn(`divnum')) => foo =>0 builtin(`define', `foo', `BAR') => foo =>BAR undefine(`foo') =>undefine(foo) foo =>BAR builtin(`undefine', `foo') => foo =>foo The NAME argument only matches the original name of the builtin, even when the `--prefix-builtins' option (or `-P', *note Invoking m4: Operation modes.) is in effect. This is different from `indir', which only tracks current macro names. $ m4 -P m4_builtin(`divnum') =>0 m4_builtin(`m4_divnum') error-->m4:stdin:2: undefined builtin `m4_divnum' => m4_indir(`divnum') error-->m4:stdin:3: undefined macro `divnum' => m4_indir(`m4_divnum') =>0 Note that `indir' and `builtin' can be used to invoke builtins without arguments, even when they normally require parameters to be recognized; but it will provoke a warning, and result in a void expansion. builtin =>builtin builtin() error-->m4:stdin:2: undefined builtin `' => builtin(`builtin') error-->m4:stdin:3: Warning: too few arguments to builtin `builtin' => builtin(`builtin',) error-->m4:stdin:4: undefined builtin `' => 6 Conditionals, loops, and recursion ************************************ Macros, expanding to plain text, perhaps with arguments, are not quite enough. We would like to have macros expand to different things, based on decisions taken at run-time. For that, we need some kind of conditionals. Also, we would like to have some kind of loop construct, so we could do something a number of times, or while some condition is true. 6.1 Testing if a macro is defined ================================= There are two different builtin conditionals in `m4'. The first is `ifdef': -- Builtin: ifdef (NAME, STRING-1, [STRING-2]) If NAME is defined as a macro, `ifdef' expands to STRING-1, otherwise to STRING-2. If STRING-2 is omitted, it is taken to be the empty string (according to the normal rules). The macro `ifdef' is recognized only with parameters. ifdef(`foo', ``foo' is defined', ``foo' is not defined') =>foo is not defined define(`foo', `') => ifdef(`foo', ``foo' is defined', ``foo' is not defined') =>foo is defined ifdef(`no_such_macro', `yes', `no', `extra argument') error-->m4:stdin:4: Warning: excess arguments to builtin `ifdef' ignored =>no 6.2 If-else construct, or multibranch ===================================== The other conditional, `ifelse', is much more powerful. It can be used as a way to introduce a long comment, as an if-else construct, or as a multibranch, depending on the number of arguments supplied: -- Builtin: ifelse (COMMENT) -- Builtin: ifelse (STRING-1, STRING-2, EQUAL, [NOT-EQUAL]) -- Builtin: ifelse (STRING-1, STRING-2, EQUAL-1, STRING-3, STRING-4, EQUAL-2, ..., [NOT-EQUAL]) Used with only one argument, the `ifelse' simply discards it and produces no output. If called with three or four arguments, `ifelse' expands into EQUAL, if STRING-1 and STRING-2 are equal (character for character), otherwise it expands to NOT-EQUAL. A final fifth argument is ignored, after triggering a warning. If called with six or more arguments, and STRING-1 and STRING-2 are equal, `ifelse' expands into EQUAL-1, otherwise the first three arguments are discarded and the processing starts again. The macro `ifelse' is recognized only with parameters. Using only one argument is a common `m4' idiom for introducing a block comment, as an alternative to repeatedly using `dnl'. This special usage is recognized by GNU `m4', so that in this case, the warning about missing arguments is never triggered. ifelse(`some comments') => ifelse(`foo', `bar') error-->m4:stdin:2: Warning: too few arguments to builtin `ifelse' => Using three or four arguments provides decision points. ifelse(`foo', `bar', `true') => ifelse(`foo', `foo', `true') =>true define(`foo', `bar') => ifelse(foo, `bar', `true', `false') =>true ifelse(foo, `foo', `true', `false') =>false Notice how the first argument was used unquoted; it is common to compare the expansion of a macro with a string. With this macro, you can now reproduce the behavior of blind builtins, where the macro is recognized only with arguments. define(`foo', `ifelse(`$#', `0', ``$0'', `arguments:$#')') => foo =>foo foo() =>arguments:1 foo(`a', `b', `c') =>arguments:3 However, `ifelse' can take more than four arguments. If given more than four arguments, `ifelse' works like a `case' or `switch' statement in traditional programming languages. If STRING-1 and STRING-2 are equal, `ifelse' expands into EQUAL-1, otherwise the procedure is repeated with the first three arguments discarded. This calls for an example: ifelse(`foo', `bar', `third', `gnu', `gnats') error-->m4:stdin:1: Warning: excess arguments to builtin `ifelse' ignored =>gnu ifelse(`foo', `bar', `third', `gnu', `gnats', `sixth') => ifelse(`foo', `bar', `third', `gnu', `gnats', `sixth', `seventh') =>seventh ifelse(`foo', `bar', `3', `gnu', `gnats', `6', `7', `8') error-->m4:stdin:4: Warning: excess arguments to builtin `ifelse' ignored =>7 Naturally, the normal case will be slightly more advanced than these examples. A common use of `ifelse' is in macros implementing loops of various kinds. 6.3 Recursion in `m4' ===================== There is no direct support for loops in `m4', but macros can be recursive. There is no limit on the number of recursion levels, other than those enforced by your hardware and operating system. Loops can be programmed using recursion and the conditionals described previously. There is a builtin macro, `shift', which can, among other things, be used for iterating through the actual arguments to a macro: -- Builtin: shift (ARG1, ...) Takes any number of arguments, and expands to all its arguments except ARG1, separated by commas, with each argument quoted. The macro `shift' is recognized only with parameters. shift =>shift shift(`bar') => shift(`foo', `bar', `baz') =>bar,baz An example of the use of `shift' is this macro: -- Composite: reverse (...) Takes any number of arguments, and reverses their order. It is implemented as: define(`reverse', `ifelse(`$#', `0', , `$#', `1', ``$1'', `reverse(shift($@)), `$1'')') => reverse => reverse(`foo') =>foo reverse(`foo', `bar', `gnats', `and gnus') =>and gnus, gnats, bar, foo While not a very interesting macro, it does show how simple loops can be made with `shift', `ifelse' and recursion. It also shows that `shift' is usually used with `$@'. Another example of this is an implementation of a short-circuiting conditional operator. -- Composite: cond (TEST-1, STRING-1, EQUAL-1, [TEST-2], [STRING-2], [EQUAL-2], ..., [NOT-EQUAL]) Similar to `ifelse', where an equal comparison between the first two strings results in the third, otherwise the first three arguments are discarded and the process repeats. The difference is that each TEST- is expanded only when it is encountered. This means that every third argument to `cond' is normally given one more level of quoting than the corresponding argument to `ifelse'. Here is the implementation of `cond', along with a demonstration of how it can short-circuit the side effects in `side'. Notice how all the unquoted side effects happen regardless of how many comparisons are made with `ifelse', compared with only the relevant effects with `cond'. define(`cond', `ifelse(`$#', `1', `$1', `ifelse($1, `$2', `$3', `$0(shift(shift(shift($@))))')')')dnl define(`side', `define(`counter', incr(counter))$1')dnl define(`example1', `define(`counter', `0')dnl ifelse(side(`$1'), `yes', `one comparison: ', side(`$1'), `no', `two comparisons: ', side(`$1'), `maybe', `three comparisons: ', `side(`default answer: ')')counter')dnl define(`example2', `define(`counter', `0')dnl cond(`side(`$1')', `yes', `one comparison: ', `side(`$1')', `no', `two comparisons: ', `side(`$1')', `maybe', `three comparisons: ', `side(`default answer: ')')counter')dnl example1(`yes') =>one comparison: 3 example1(`no') =>two comparisons: 3 example1(`maybe') =>three comparisons: 3 example1(`feeling rather indecisive today') =>default answer: 4 example2(`yes') =>one comparison: 1 example2(`no') =>two comparisons: 2 example2(`maybe') =>three comparisons: 3 example2(`feeling rather indecisive today') =>default answer: 4 Another common task that requires iteration is joining a list of arguments into a single string. -- Composite: join ([SEPARATOR], [ARGS...]) -- Composite: joinall ([SEPARATOR], [ARGS...]) Generate a single-quoted string, consisting of each ARG separated by SEPARATOR. While `joinall' always outputs a SEPARATOR between arguments, `join' avoids the SEPARATOR for an empty ARG. Here are some examples of its usage, based on the implementation `m4-1.4.11/examples/join.m4' distributed in this package: $ m4 -I examples include(`join.m4') => join,join(`-'),join(`-', `'),join(`-', `', `') =>,,, joinall,joinall(`-'),joinall(`-', `'),joinall(`-', `', `') =>,,,- join(`-', `1') =>1 join(`-', `1', `2', `3') =>1-2-3 join(`', `1', `2', `3') =>123 join(`-', `', `1', `', `', `2', `') =>1-2 joinall(`-', `', `1', `', `', `2', `') =>-1---2- join(`,', `1', `2', `3') =>1,2,3 define(`nargs', `$#')dnl nargs(join(`,', `1', `2', `3')) =>1 Examining the implementation shows some interesting points about several m4 programming idioms. $ m4 -I examples undivert(`join.m4')dnl =>divert(`-1') =># join(sep, args) - join each non-empty ARG into a single =># string, with each element separated by SEP =>define(`join', =>`ifelse(`$#', `2', ``$2'', => `ifelse(`$2', `', `', ``$2'_')$0(`$1', shift(shift($@)))')') =>define(`_join', =>`ifelse(`$#$2', `2', `', => `ifelse(`$2', `', `', ``$1$2'')$0(`$1', shift(shift($@)))')') =># joinall(sep, args) - join each ARG, including empty ones, =># into a single string, with each element separated by SEP =>define(`joinall', ``$2'_$0(`$1', shift($@))') =>define(`_joinall', =>`ifelse(`$#', `2', `', ``$1$3'$0(`$1', shift(shift($@)))')') =>divert`'dnl First, notice that this implementation creates helper macros `_join' and `_joinall'. This division of labor makes it easier to output the correct number of SEPARATOR instances: `join' and `joinall' are responsible for the first argument, without a separator, while `_join' and `_joinall' are responsible for all remaining arguments, always outputting a separator when outputting an argument. Next, observe how `join' decides to iterate to itself, because the first ARG was empty, or to output the argument and swap over to `_join'. If the argument is non-empty, then the nested `ifelse' results in an unquoted `_', which is concatenated with the `$0' to form the next macro name to invoke. The `joinall' implementation is simpler since it does not have to suppress empty ARG; it always executes once then defers to `_joinall'. Another important idiom is the idea that SEPARATOR is reused for each iteration. Each iteration has one less argument, but rather than discarding `$1' by iterating with `$0(shift($@))', the macro discards `$2' by using `$0(`$1', shift(shift($@)))'. Next, notice that it is possible to compare more than one condition in a single `ifelse' test. The test of `$#$2' against `2' allows `_join' to iterate for two separate reasons--either there are still more than two arguments, or there are exactly two arguments but the last argument is not empty. Finally, notice that these macros require exactly two arguments to terminate recursion, but that they still correctly result in empty output when given no ARGS (i.e., zero or one macro argument). On the first pass when there are too few arguments, the `shift' results in no output, but leaves an empty string to serve as the required second argument for the second pass. Put another way, ``$1', shift($@)' is not the same as `$@', since only the former guarantees at least two arguments. Sometimes, a recursive algorithm requires adding quotes to each element, or treating multiple arguments as a single element: -- Composite: quote (...) -- Composite: dquote (...) -- Composite: dquote_elt (...) Takes any number of arguments, and adds quoting. With `quote', only one level of quoting is added, effectively removing whitespace after commas and turning multiple arguments into a single string. With `dquote', two levels of quoting are added, one around each element, and one around the list. And with `dquote_elt', two levels of quoting are added around each element. An actual implementation of these three macros is distributed as `m4-1.4.11/examples/quote.m4' in this package. First, let's examine their usage: $ m4 -I examples include(`quote.m4') => -quote-dquote-dquote_elt- =>---- -quote()-dquote()-dquote_elt()- =>--`'-`'- -quote(`1')-dquote(`1')-dquote_elt(`1')- =>-1-`1'-`1'- -quote(`1', `2')-dquote(`1', `2')-dquote_elt(`1', `2')- =>-1,2-`1',`2'-`1',`2'- define(`n', `$#')dnl -n(quote(`1', `2'))-n(dquote(`1', `2'))-n(dquote_elt(`1', `2'))- =>-1-1-2- dquote(dquote_elt(`1', `2')) =>``1'',``2'' dquote_elt(dquote(`1', `2')) =>``1',`2'' The last two lines show that when given two arguments, `dquote' results in one string, while `dquote_elt' results in two. Now, examine the implementation. Note that `quote' and `dquote_elt' make decisions based on their number of arguments, so that when called without arguments, they result in nothing instead of a quoted empty string; this is so that it is possible to distinguish between no arguments and an empty first argument. `dquote', on the other hand, results in a string no matter what, since it is still possible to tell whether it was invoked without arguments based on the resulting string. $ m4 -I examples undivert(`quote.m4')dnl =>divert(`-1') =># quote(args) - convert args to single-quoted string =>define(`quote', `ifelse(`$#', `0', `', ``$*'')') =># dquote(args) - convert args to quoted list of quoted strings =>define(`dquote', ``$@'') =># dquote_elt(args) - convert args to list of double-quoted strings =>define(`dquote_elt', `ifelse(`$#', `0', `', `$#', `1', ```$1''', => ```$1'',$0(shift($@))')') =>divert`'dnl It is worth pointing out that `quote(ARGS)' is more efficient than `joinall(`,', ARGS)' for producing the same output. One more useful macro based on `shift' allows portably selecting an arbitrary argument (usually greater than the ninth argument), without relying on the GNU extension of multi-digit arguments (*note Arguments::). -- Composite: argn (N, ...) Expands to argument N out of the remaining arguments. N must be a positive number. Usually invoked as `argn(`N',$@)'. It is implemented as: define(`argn', `ifelse(`$1', 1, ``$2'', `argn(decr(`$1'), shift(shift($@)))')') => argn(`1', `a') =>a define(`foo', `argn(`11', $@)') => foo(`a', `b', `c', `d', `e', `f', `g', `h', `i', `j', `k', `l') =>k 6.4 Iteration by counting ========================= Here is an example of a loop macro that implements a simple for loop. -- Composite: forloop (ITERATOR, START, END, TEXT) Takes the name in ITERATOR, which must be a valid macro name, and successively assign it each integer value from START to END, inclusive. For each assignment to ITERATOR, append TEXT to the expansion of the `forloop'. TEXT may refer to ITERATOR. Any definition of ITERATOR prior to this invocation is restored. It can, for example, be used for simple counting: $ m4 -I examples include(`forloop.m4') => forloop(`i', `1', `8', `i ') =>1 2 3 4 5 6 7 8 For-loops can be nested, like: $ m4 -I examples include(`forloop.m4') => forloop(`i', `1', `4', `forloop(`j', `1', `8', ` (i, j)') ') => (1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6) (1, 7) (1, 8) => (2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6) (2, 7) (2, 8) => (3, 1) (3, 2) (3, 3) (3, 4) (3, 5) (3, 6) (3, 7) (3, 8) => (4, 1) (4, 2) (4, 3) (4, 4) (4, 5) (4, 6) (4, 7) (4, 8) => The implementation of the `forloop' macro is fairly straightforward. The `forloop' macro itself is simply a wrapper, which saves the previous definition of the first argument, calls the internal macro `_forloop', and re-establishes the saved definition of the first argument. The macro `_forloop' expands the fourth argument once, and tests to see if the iterator has reached the final value. If it has not finished, it increments the iterator (using the predefined macro `incr', *note Incr::), and recurses. Here is an actual implementation of `forloop', distributed as `m4-1.4.11/examples/forloop.m4' in this package: $ m4 -I examples undivert(`forloop.m4')dnl =>divert(`-1') =># forloop(var, from, to, stmt) - simple version =>define(`forloop', `pushdef(`$1', `$2')_forloop($@)popdef(`$1')') =>define(`_forloop', => `$4`'ifelse($1, `$3', `', `define(`$1', incr($1))$0($@)')') =>divert`'dnl Notice the careful use of quotes. Certain macro arguments are left unquoted, each for its own reason. Try to find out _why_ these arguments are left unquoted, and see what happens if they are quoted. (As presented, these two macros are useful but not very robust for general use. They lack even basic error handling for cases like START less than END, END not numeric, or ITERATOR not being a macro name. See if you can improve these macros; or *note Answers: Improved forloop.). 6.5 Iteration by list contents ============================== Here is an example of a loop macro that implements list iteration. -- Composite: foreach (ITERATOR, PAREN-LIST, TEXT) -- Composite: foreachq (ITERATOR, QUOTE-LIST, TEXT) Takes the name in ITERATOR, which must be a valid macro name, and successively assign it each value from PAREN-LIST or QUOTE-LIST. In `foreach', PAREN-LIST is a comma-separated list of elements contained in parentheses. In `foreachq', QUOTE-LIST is a comma-separated list of elements contained in a quoted string. For each assignment to ITERATOR, append TEXT to the overall expansion. TEXT may refer to ITERATOR. Any definition of ITERATOR prior to this invocation is restored. As an example, this displays each word in a list inside of a sentence, using an implementation of `foreach' distributed as `m4-1.4.11/examples/foreach.m4', and `foreachq' in `m4-1.4.11/examples/foreachq.m4'. $ m4 -I examples include(`foreach.m4') => foreach(`x', (foo, bar, foobar), `Word was: x ')dnl =>Word was: foo =>Word was: bar =>Word was: foobar include(`foreachq.m4') => foreachq(`x', `foo, bar, foobar', `Word was: x ')dnl =>Word was: foo =>Word was: bar =>Word was: foobar It is possible to be more complex; each element of the PAREN-LIST or QUOTE-LIST can itself be a list, to pass as further arguments to a helper macro. This example generates a shell case statement: $ m4 -I examples include(`foreach.m4') => define(`_case', ` $1) $2=" $1";; ')dnl define(`_cat', `$1$2')dnl case $`'1 in =>case $1 in foreach(`x', `(`(`a', `vara')', `(`b', `varb')', `(`c', `varc')')', `_cat(`_case', x)')dnl => a) => vara=" a";; => b) => varb=" b";; => c) => varc=" c";; esac =>esac The implementation of the `foreach' macro is a bit more involved; it is a wrapper around two helper macros. First, `_arg1' is needed to grab the first element of a list. Second, `_foreach' implements the recursion, successively walking through the original list. Here is a simple implementation of `foreach': $ m4 -I examples undivert(`foreach.m4')dnl =>divert(`-1') =># foreach(x, (item_1, item_2, ..., item_n), stmt) =># parenthesized list, simple version =>define(`foreach', `pushdef(`$1')_foreach($@)popdef(`$1')') =>define(`_arg1', `$1') =>define(`_foreach', `ifelse(`$2', `()', `', => `define(`$1', _arg1$2)$3`'$0(`$1', (shift$2), `$3')')') =>divert`'dnl Unfortunately, that implementation is not robust to macro names as list elements. Each iteration of `_foreach' is stripping another layer of quotes, leading to erratic results if list elements are not already fully expanded. The first cut at implementing `foreachq' takes this into account. Also, when using quoted elements in a PAREN-LIST, the overall list must be quoted. A QUOTE-LIST has the nice property of requiring fewer characters to create a list containing the same quoted elements. To see the difference between the two macros, we attempt to pass double-quoted macro names in a list, expecting the macro name on output after one layer of quotes is removed during list iteration and the final layer removed during the final rescan: $ m4 -I examples define(`a', `1')define(`b', `2')define(`c', `3') => include(`foreach.m4') => include(`foreachq.m4') => foreach(`x', `(``a'', ``(b'', ``c)'')', `x ') =>1 =>(2)1 => =>, x =>) foreachq(`x', ```a'', ``(b'', ``c)''', `x ')dnl =>a =>(b =>c) Obviously, `foreachq' did a better job; here is its implementation: $ m4 -I examples undivert(`foreachq.m4')dnl =>include(`quote.m4')dnl =>divert(`-1') =># foreachq(x, `item_1, item_2, ..., item_n', stmt) =># quoted list, simple version =>define(`foreachq', `pushdef(`$1')_foreachq($@)popdef(`$1')') =>define(`_arg1', `$1') =>define(`_foreachq', `ifelse(quote($2), `', `', => `define(`$1', `_arg1($2)')$3`'$0(`$1', `shift($2)', `$3')')') =>divert`'dnl Notice that `_foreachq' had to use the helper macro `quote' defined earlier (*note Shift::), to ensure that the embedded `ifelse' call does not go haywire if a list element contains a comma. Unfortunately, this implementation of `foreachq' has its own severe flaw. Whereas the `foreach' implementation was linear, this macro is quadratic in the number of list elements, and is much more likely to trip up the limit set by the command line option `--nesting-limit' (or `-L', *note Invoking m4: Limits control.). Additionally, this implementation does not expand `defn(`ITERATOR')' very well, when compared with `foreach'. $ m4 -I examples include(`foreach.m4')include(`foreachq.m4') => foreach(`name', `(`a', `b')', ` defn(`name')') => a b foreachq(`name', ``a', `b'', ` defn(`name')') => _arg1(`a', `b') _arg1(shift(`a', `b')) It is possible to have robust iteration with linear behavior and sane ITERATOR contents for either list style. See if you can learn from the best elements of both of these implementations to create robust macros (or *note Answers: Improved foreach.). 7 How to debug macros and input ******************************* When writing macros for `m4', they often do not work as intended on the first try (as is the case with most programming languages). Fortunately, there is support for macro debugging in `m4'. 7.1 Displaying macro definitions ================================ If you want to see what a name expands into, you can use the builtin `dumpdef': -- Builtin: dumpdef ([NAMES...]) Accepts any number of arguments. If called without any arguments, it displays the definitions of all known names, otherwise it displays the definitions of the NAMES given. The output is printed to the current debug file (usually standard error), and is sorted by name. If an unknown name is encountered, a warning is printed. The expansion of `dumpdef' is void. $ m4 -d define(`foo', `Hello world.') => dumpdef(`foo') error-->foo: `Hello world.' => dumpdef(`define') error-->define: => The last example shows how builtin macros definitions are displayed. The definition that is dumped corresponds to what would occur if the macro were to be called at that point, even if other definitions are still live due to redefining a macro during argument collection. $ m4 -d pushdef(`f', ``$0'1')pushdef(`f', ``$0'2') => f(popdef(`f')dumpdef(`f')) error-->f: ``$0'1' =>f2 f(popdef(`f')dumpdef(`f')) error-->m4:stdin:3: undefined macro `f' =>f1 *Note Debug Levels::, for information on controlling the details of the display. 7.2 Tracing macro calls ======================= It is possible to trace macro calls and expansions through the builtins `traceon' and `traceoff': -- Builtin: traceon ([NAMES...]) -- Builtin: traceoff ([NAMES...]) When called without any arguments, `traceon' and `traceoff' will turn tracing on and off, respectively, for all currently defined macros. When called with arguments, only the macros listed in NAMES are affected, whether or not they are currently defined. The expansion of `traceon' and `traceoff' is void. Whenever a traced macro is called and the arguments have been collected, the call is displayed. If the expansion of the macro call is not void, the expansion can be displayed after the call. The output is printed to the current debug file (defaulting to standard error, *note Debug Output::). $ m4 -d define(`foo', `Hello World.') => define(`echo', `$@') => traceon(`foo', `echo') => foo error-->m4trace: -1- foo -> `Hello World.' =>Hello World. echo(`gnus', `and gnats') error-->m4trace: -1- echo(`gnus', `and gnats') -> ``gnus',`and gnats'' =>gnus,and gnats The number between dashes is the depth of the expansion. It is one most of the time, signifying an expansion at the outermost level, but it increases when macro arguments contain unquoted macro calls. The maximum number that will appear between dashes is controlled by the option `--nesting-limit' (or `-L', *note Invoking m4: Limits control.). Additionally, the option `--trace' (or `-t') can be used to invoke `traceon(NAME)' before parsing input. $ m4 -L 3 -t ifelse ifelse(`one level') error-->m4trace: -1- ifelse => ifelse(ifelse(ifelse(`three levels'))) error-->m4trace: -3- ifelse error-->m4trace: -2- ifelse error-->m4trace: -1- ifelse => ifelse(ifelse(ifelse(ifelse(`four levels')))) error-->m4:stdin:3: recursion limit of 3 exceeded, use -L to change it Tracing by name is an attribute that is preserved whether the macro is defined or not. This allows the selection of macros to trace before those macros are defined. $ m4 -d traceoff(`foo') => traceon(`foo') => foo =>foo define(`foo', `bar') => foo error-->m4trace: -1- foo -> `bar' =>bar undefine(`foo') => ifdef(`foo', `yes', `no') =>no indir(`foo') error-->m4:stdin:8: undefined macro `foo' => define(`foo', `blah') => foo error-->m4trace: -1- foo -> `blah' =>blah traceoff => foo =>blah Tracing even works on builtins. However, `defn' (*note Defn::) does not transfer tracing status. $ m4 -d traceon(`eval', `m4_divnum') => define(`m4_eval', defn(`eval')) => define(`m4_divnum', defn(`divnum')) => eval(divnum) error-->m4trace: -1- eval(`0') -> `0' =>0 m4_eval(m4_divnum) error-->m4trace: -2- m4_divnum -> `0' =>0 *Note Debug Levels::, for information on controlling the details of the display. 7.3 Controlling debugging output ================================ The `-d' option to `m4' (or `--debug', *note Invoking m4: Debugging options.) controls the amount of details presented in three categories of output. Trace output is requested by `traceon' (*note Trace::), and each line is prefixed by `m4trace:' in relation to a macro invocation. Debug output tracks useful events not associated with a macro invocation, and each line is prefixed by `m4debug:'. Finally, `dumpdef' (*note Dumpdef::) output is affected, with no prefix added to the output lines. The FLAGS following the option can be one or more of the following: `a' In trace output, show the actual arguments that were collected before invoking the macro. This applies to all macro calls if the `t' flag is used, otherwise only the macros covered by calls of `traceon'. `c' In trace output, show several trace lines for each macro call. A line is shown when the macro is seen, but before the arguments are collected; a second line when the arguments have been collected and a third line after the call has completed. `e' In trace output, show the expansion of each macro call, if it is not void. This applies to all macro calls if the `t' flag is used, otherwise only the macros covered by calls of `traceon'. `f' In debug and trace output, include the name of the current input file in the output line. `i' In debug output, print a message each time the current input file is changed. `l' In debug and trace output, include the current input line number in the output line. `p' In debug output, print a message when a named file is found through the path search mechanism (*note Search Path::), giving the actual file name used. `q' In trace and dumpdef output, quote actual arguments and macro expansions in the display with the current quotes. This is useful in connection with the `a' and `e' flags above. `t' In trace output, trace all macro calls made in this invocation of `m4', regardless of the settings of `traceon'. `x' In trace output, add a unique `macro call id' to each line of the trace output. This is useful in connection with the `c' flag above. `V' A shorthand for all of the above flags. If no flags are specified with the `-d' option, the default is `aeq'. The examples throughout this manual assume the default flags. There is a builtin macro `debugmode', which allows on-the-fly control of the debugging output format: -- Builtin: debugmode ([FLAGS]) The argument FLAGS should be a subset of the letters listed above. As special cases, if the argument starts with a `+', the flags are added to the current debug flags, and if it starts with a `-', they are removed. If no argument is present, all debugging flags are cleared (as if no `-d' was given), and with an empty argument the flags are reset to the default of `aeq'. The expansion of `debugmode' is void. $ m4 define(`foo', `FOO') => traceon(`foo') => debugmode() => foo error-->m4trace: -1- foo -> `FOO' =>FOO debugmode => foo error-->m4trace: -1- foo =>FOO debugmode(`+l') => foo error-->m4trace:8: -1- foo =>FOO 7.4 Saving debugging output =========================== Debug and tracing output can be redirected to files using either the `--debugfile' option to `m4' (*note Invoking m4: Debugging options.), or with the builtin macro `debugfile': -- Builtin: debugfile ([FILE]) Sends all further debug and trace output to FILE, opened in append mode. If FILE is the empty string, debug and trace output are discarded. If `debugfile' is called without any arguments, debug and trace output are sent to standard error. This does not affect warnings, error messages, or `errprint' output, which are always sent to standard error. If FILE cannot be opened, the current debug file is unchanged, and an error is issued. The expansion of `debugfile' is void. $ m4 -d traceon(`divnum') => divnum(`extra') error-->m4:stdin:2: Warning: excess arguments to builtin `divnum' ignored error-->m4trace: -1- divnum(`extra') -> `0' =>0 debugfile() => divnum(`extra') error-->m4:stdin:4: Warning: excess arguments to builtin `divnum' ignored =>0 debugfile => divnum error-->m4trace: -1- divnum -> `0' =>0 8 Input control *************** This chapter describes various builtin macros for controlling the input to `m4'. 8.1 Deleting whitespace in input ================================ The builtin `dnl' stands for "Discard to Next Line": -- Builtin: dnl All characters, up to and including the next newline, are discarded without performing any macro expansion. A warning is issued if the end of the file is encountered without a newline. The expansion of `dnl' is void. It is often used in connection with `define', to remove the newline that follows the call to `define'. Thus define(`foo', `Macro `foo'.')dnl A very simple macro, indeed. foo =>Macro foo. The input up to and including the next newline is discarded, as opposed to the way comments are treated (*note Comments::). Usually, `dnl' is immediately followed by an end of line or some other whitespace. GNU `m4' will produce a warning diagnostic if `dnl' is followed by an open parenthesis. In this case, `dnl' will collect and process all arguments, looking for a matching close parenthesis. All predictable side effects resulting from this collection will take place. `dnl' will return no output. The input following the matching close parenthesis up to and including the next newline, on whatever line containing it, will still be discarded. dnl(`args are ignored, but side effects occur', define(`foo', `like this')) while this text is ignored: undefine(`foo') error-->m4:stdin:1: Warning: excess arguments to builtin `dnl' ignored See how `foo' was defined, foo? =>See how foo was defined, like this? If the end of file is encountered without a newline character, a warning is issued and dnl stops consuming input. m4wrap(`m4wrap(`2 hi ')0 hi dnl 1 hi') => define(`hi', `HI') => ^D error-->m4:stdin:1: Warning: end of file treated as newline =>0 HI 2 HI 8.2 Changing the quote characters ================================= The default quote delimiters can be changed with the builtin `changequote': -- Builtin: changequote ([START = ``'], [END = `'']) This sets START as the new begin-quote delimiter and END as the new end-quote delimiter. If both arguments are missing, the default quotes (``' and `'') are used. If START is void, then quoting is disabled. Otherwise, if END is missing or void, the default end-quote delimiter (`'') is used. The quote delimiters can be of any length. The expansion of `changequote' is void. changequote(`[', `]') => define([foo], [Macro [foo].]) => foo =>Macro foo. The quotation strings can safely contain eight-bit characters. If no single character is appropriate, START and END can be of any length. Other implementations cap the delimiter length to five characters, but GNU has no inherent limit. changequote(`[[[', `]]]') => define([[[foo]]], [[[Macro [[[[[foo]]]]].]]]) => foo =>Macro [[foo]]. Calling `changequote' with START as the empty string will effectively disable the quoting mechanism, leaving no way to quote text. However, using an empty string is not portable, as some other implementations of `m4' revert to the default quoting, while others preserve the prior non-empty delimiter. If START is not empty, then an empty END will use the default end-quote delimiter of `'', as otherwise, it would be impossible to end a quoted string. Again, this is not portable, as some other `m4' implementations reuse START as the end-quote delimiter, while others preserve the previous non-empty value. Omitting both arguments restores the default begin-quote and end-quote delimiters; fortunately this behavior is portable to all implementations of `m4'. define(`foo', `Macro `FOO'.') => changequote(`', `') => foo =>Macro `FOO'. `foo' =>`Macro `FOO'.' changequote(`,) => foo =>Macro FOO. There is no way in `m4' to quote a string containing an unmatched begin-quote, except using `changequote' to change the current quotes. If the quotes should be changed from, say, `[' to `[[', temporary quote characters have to be defined. To achieve this, two calls of `changequote' must be made, one for the temporary quotes and one for the new quotes. Macros are recognized in preference to the begin-quote string, so if a prefix of START can be recognized as part of a potential macro name, the quoting mechanism is effectively disabled. Unless you use `changeword' (*note Changeword::), this means that START should not begin with a letter, digit, or `_' (underscore). However, even though quoted strings are not recognized, the quote characters can still be discerned in macro expansion and in trace output. define(`echo', `$@') => define(`hi', `HI') => changequote(`q', `Q') => q hi Q hi =>q HI Q HI echo(hi) =>qHIQ changequote => changequote(`-', `EOF') => - hi EOF hi => hi HI changequote => changequote(`1', `2') => hi1hi2 =>hi1hi2 hi 1hi2 =>HI hi Quotes are recognized in preference to argument collection. In particular, if START is a single `(', then argument collection is effectively disabled. For portability with other implementations, it is a good idea to avoid `(', `,', and `)' as the first character in START. define(`echo', `$#:$@:') => define(`hi', `HI') => changequote(`(',`)') => echo(hi) =>0::hi changequote => changequote(`((', `))') => echo(hi) =>1:HI: echo((hi)) =>0::hi changequote => changequote(`,', `)') => echo(hi,hi)bye) =>1:HIhibye: However, if you are not worried about portability, using `(' and `)' as quoting characters has an interesting property--you can use it to compute a quoted string containing the expansion of any quoted text, as long as the expansion results in both balanced quotes and balanced parentheses. The trick is realizing `expand' uses `$1' unquoted, to trigger its expansion using the normal quoting characters, but uses extra parentheses to group unquoted commas that occur in the expansion without consuming whitespace following those commas. Then `_expand' uses `changequote' to convert the extra parentheses back into quoting characters. Note that it takes two more `changequote' invocations to restore the original quotes. Contrast the behavior on whitespace when using `$*', via `quote', to attempt the same task. changequote(`[', `]')dnl define([a], [1, (b)])dnl define([b], [2])dnl define([quote], [[$*]])dnl define([expand], [_$0(($1))])dnl define([_expand], [changequote([(], [)])$1changequote`'changequote(`[', `]')])dnl expand([a, a, [a, a], [[a, a]]]) =>1, (2), 1, (2), a, a, [a, a] quote(a, a, [a, a], [[a, a]]) =>1,(2),1,(2),a, a,[a, a] If END is a prefix of START, the end-quote will be recognized in preference to a nested begin-quote. In particular, changing the quotes to have the same string for START and END disables nesting of quotes. When quote nesting is disabled, it is impossible to double-quote strings across macro expansions, so using the same string is not done very often. define(`hi', `HI') => changequote(`""', `"') => ""hi"""hi" =>hihi ""hi" ""hi" =>hi hi ""hi"" "hi" =>hi" "HI" changequote => `hi`hi'hi' =>hi`hi'hi changequote(`"', `"') => "hi"hi"hi" =>hiHIhi It is an error if the end of file occurs within a quoted string. `hello world' =>hello world `dangling quote ^D error-->m4:stdin:2: ERROR: end of file in string 8.3 Changing the comment delimiters =================================== The default comment delimiters can be changed with the builtin macro `changecom': -- Builtin: changecom ([START], [END = `']) This sets START as the new begin-comment delimiter and END as the new end-comment delimiter. If both arguments are missing, or START is void, then comments are disabled. Otherwise, if END is missing or void, the default end-comment delimiter of newline is used. The comment delimiters can be of any length. The expansion of `changecom' is void. define(`comment', `COMMENT') => # A normal comment =># A normal comment changecom(`/*', `*/') => # Not a comment anymore =># Not a COMMENT anymore But: /* this is a comment now */ while this is not a comment =>But: /* this is a comment now */ while this is not a COMMENT Note how comments are copied to the output, much as if they were quoted strings. If you want the text inside a comment expanded, quote the begin-comment delimiter. Calling `changecom' without any arguments, or with START as the empty string, will effectively disable the commenting mechanism. To restore the original comment start of `#', you must explicitly ask for it. If START is not empty, then an empty END will use the default end-comment delimiter of newline, as otherwise, it would be impossible to end a comment. However, this is not portable, as some other `m4' implementations preserve the previous non-empty delimiters instead. define(`comment', `COMMENT') => changecom => # Not a comment anymore =># Not a COMMENT anymore changecom(`#', `') => # comment again =># comment again The comment strings can safely contain eight-bit characters. If no single character is appropriate, START and END can be of any length. Other implementations cap the delimiter length to five characters, but GNU has no inherent limit. Comments are recognized in preference to macros. However, this is not compatible with other implementations, where macros and even quoting takes precedence over comments, so it may change in a future release. For portability, this means that START should not begin with a letter, digit, or `_' (underscore), and that neither the start-quote nor the start-comment string should be a prefix of the other. define(`hi', `HI') => define(`hi1hi2', `hello') => changecom(`q', `Q') => q hi Q hi =>q hi Q HI changecom(`1', `2') => hi1hi2 =>hello hi 1hi2 =>HI 1hi2 Comments are recognized in preference to argument collection. In particular, if START is a single `(', then argument collection is effectively disabled. For portability with other implementations, it is a good idea to avoid `(', `,', and `)' as the first character in START. define(`echo', `$#:$*:$@:') => define(`hi', `HI') => changecom(`(',`)') => echo(hi) =>0:::(hi) changecom => changecom(`((', `))') => echo(hi) =>1:HI:HI: echo((hi)) =>0:::((hi)) changecom(`,', `)') => echo(hi,hi)bye) =>1:HI,hi)bye:HI,hi)bye: changecom => echo(hi,`,`'hi',hi) =>3:HI,,HI,HI:HI,,`'hi,HI: echo(hi,`,`'hi',hi`'changecom(`,,', `hi')) =>3:HI,,`'hi,HI:HI,,`'hi,HI: It is an error if the end of file occurs within a comment. changecom(`/*', `*/') => /*dangling comment ^D error-->m4:stdin:2: ERROR: end of file in comment 8.4 Changing the lexical structure of words =========================================== The macro `changeword' and all associated functionality is experimental. It is only available if the `--enable-changeword' option was given to `configure', at GNU `m4' installation time. The functionality will go away in the future, to be replaced by other new features that are more efficient at providing the same capabilities. _Do not rely on it_. Please direct your comments about it the same way you would do for bugs. A file being processed by `m4' is split into quoted strings, words (potential macro names) and simple tokens (any other single character). Initially a word is defined by the following regular expression: [_a-zA-Z][_a-zA-Z0-9]* Using `changeword', you can change this regular expression: -- Optional builtin: changeword (REGEX) Changes the regular expression for recognizing macro names to be REGEX. If REGEX is empty, use `[_a-zA-Z][_a-zA-Z0-9]*'. REGEX must obey the constraint that every prefix of the desired final pattern is also accepted by the regular expression. If REGEX contains grouping parentheses, the macro invoked is the portion that matched the first group, rather than the entire matching string. The expansion of `changeword' is void. The macro `changeword' is recognized only with parameters. Relaxing the lexical rules of `m4' might be useful (for example) if you wanted to apply translations to a file of numbers: ifdef(`changeword', `', `errprint(` skipping: no changeword support ')m4exit(`77')')dnl changeword(`[_a-zA-Z0-9]+') => define(`1', `0')1 =>0 Tightening the lexical rules is less useful, because it will generally make some of the builtins unavailable. You could use it to prevent accidental call of builtins, for example: ifdef(`changeword', `', `errprint(` skipping: no changeword support ')m4exit(`77')')dnl define(`_indir', defn(`indir')) => changeword(`_[_a-zA-Z0-9]*') => esyscmd(`foo') =>esyscmd(foo) _indir(`esyscmd', `echo hi') =>hi => Because `m4' constructs its words a character at a time, there is a restriction on the regular expressions that may be passed to `changeword'. This is that if your regular expression accepts `foo', it must also accept `f' and `fo'. ifdef(`changeword', `', `errprint(` skipping: no changeword support ')m4exit(`77')')dnl define(`foo ', `bar ') => dnl This example wants to recognize changeword, dnl, and `foo\n'. dnl First, we check that our regexp will match. regexp(`changeword', `[cd][a-z]*\|foo[ ]') =>0 regexp(`foo ', `[cd][a-z]*\|foo[ ]') =>0 regexp(`f', `[cd][a-z]*\|foo[ ]') =>-1 foo =>foo changeword(`[cd][a-z]*\|foo[ ]') => dnl Even though `foo\n' matches, we forgot to allow `f'. foo =>foo changeword(`[cd][a-z]*\|fo*[ ]?') => dnl Now we can call `foo\n'. foo =>bar `changeword' has another function. If the regular expression supplied contains any grouped subexpressions, then text outside the first of these is discarded before symbol lookup. So: ifdef(`changeword', `', `errprint(` skipping: no changeword support ')m4exit(`77')')dnl ifdef(`__unix__', , `errprint(` skipping: syscmd does not have unix semantics ')m4exit(`77')')dnl changecom(`/*', `*/')dnl define(`foo', `bar')dnl changeword(`#\([_a-zA-Z0-9]*\)') => #esyscmd(`echo foo \#foo') =>foo bar => `m4' now requires a `#' mark at the beginning of every macro invocation, so one can use `m4' to preprocess plain text without losing various words like `divert'. In `m4', macro substitution is based on text, while in TeX, it is based on tokens. `changeword' can throw this difference into relief. For example, here is the same idea represented in TeX and `m4'. First, the TeX version: \def\a{\message{Hello}} \catcode`\@=0 \catcode`\\=12 @a @bye =>Hello Then, the `m4' version: ifdef(`changeword', `', `errprint(` skipping: no changeword support ')m4exit(`77')')dnl define(`a', `errprint(`Hello')')dnl changeword(`@\([_a-zA-Z0-9]*\)') => @a =>errprint(Hello) In the TeX example, the first line defines a macro `a' to print the message `Hello'. The second line defines <@> to be usable instead of <\> as an escape character. The third line defines <\> to be a normal printing character, not an escape. The fourth line invokes the macro `a'. So, when TeX is run on this file, it displays the message `Hello'. When the `m4' example is passed through `m4', it outputs `errprint(Hello)'. The reason for this is that TeX does lexical analysis of macro definition when the macro is _defined_. `m4' just stores the text, postponing the lexical analysis until the macro is _used_. You should note that using `changeword' will slow `m4' down by a factor of about seven, once it is changed to something other than the default regular expression. You can invoke `changeword' with the empty string to restore the default word definition, and regain the parsing speed. 8.5 Saving text until end of input ================================== It is possible to `save' some text until the end of the normal input has been seen. Text can be saved, to be read again by `m4' when the normal input has been exhausted. This feature is normally used to initiate cleanup actions before normal exit, e.g., deleting temporary files. To save input text, use the builtin `m4wrap': -- Builtin: m4wrap (STRING, ...) Stores STRING in a safe place, to be reread when end of input is reached. As a GNU extension, additional arguments are concatenated with a space to the STRING. The expansion of `m4wrap' is void. The macro `m4wrap' is recognized only with parameters. define(`cleanup', `This is the `cleanup' action. ') => m4wrap(`cleanup') => This is the first and last normal input line. =>This is the first and last normal input line. ^D =>This is the cleanup action. The saved input is only reread when the end of normal input is seen, and not if `m4exit' is used to exit `m4'. It is safe to call `m4wrap' from saved text, but then the order in which the saved text is reread is undefined. If `m4wrap' is not used recursively, the saved pieces of text are reread in the opposite order in which they were saved (LIFO--last in, first out). However, this behavior is likely to change in a future release, to match POSIX, so you should not depend on this order. It is possible to emulate POSIX behavior even with older versions of GNU M4 by including the file `m4-1.4.11/examples/wrapfifo.m4' from the distribution: $ m4 -I examples undivert(`wrapfifo.m4')dnl =>dnl Redefine m4wrap to have FIFO semantics. =>define(`_m4wrap_level', `0')dnl =>define(`m4wrap', =>`ifdef(`m4wrap'_m4wrap_level, => `define(`m4wrap'_m4wrap_level, => defn(`m4wrap'_m4wrap_level)`$1')', => `builtin(`m4wrap', `define(`_m4wrap_level', => incr(_m4wrap_level))dnl =>m4wrap'_m4wrap_level)dnl =>define(`m4wrap'_m4wrap_level, `$1')')')dnl include(`wrapfifo.m4') => m4wrap(`a`'m4wrap(`c ', `d')')m4wrap(`b') => ^D =>abc It is likewise possible to emulate LIFO behavior without resorting to the GNU M4 extension of `builtin', by including the file `m4-1.4.11/examples/wraplifo.m4' from the distribution. (Unfortunately, both examples shown here share some subtle bugs. See if you can find and correct them; or *note Answers: Improved m4wrap.). $ m4 -I examples undivert(`wraplifo.m4')dnl =>dnl Redefine m4wrap to have LIFO semantics. =>define(`_m4wrap_level', `0')dnl =>define(`_m4wrap', defn(`m4wrap'))dnl =>define(`m4wrap', =>`ifdef(`m4wrap'_m4wrap_level, => `define(`m4wrap'_m4wrap_level, => `$1'defn(`m4wrap'_m4wrap_level))', => `_m4wrap(`define(`_m4wrap_level', incr(_m4wrap_level))dnl =>m4wrap'_m4wrap_level)dnl =>define(`m4wrap'_m4wrap_level, `$1')')')dnl include(`wraplifo.m4') => m4wrap(`a`'m4wrap(`c ', `d')')m4wrap(`b') => ^D =>bac Here is an example of implementing a factorial function using `m4wrap': define(`f', `ifelse(`$1', `0', `Answer: 0!=1 ', eval(`$1>1'), `0', `Answer: $2$1=eval(`$2$1') ', `m4wrap(`f(decr(`$1'), `$2$1*')')')') => f(`10') => ^D =>Answer: 10*9*8*7*6*5*4*3*2*1=3628800 Invocations of `m4wrap' at the same recursion level are concatenated and rescanned as usual: define(`aa', `AA ') => m4wrap(`a')m4wrap(`a') => ^D =>AA however, the transition between recursion levels behaves like an end of file condition between two input files. m4wrap(`m4wrap(`)')len(abc') => ^D error-->m4:stdin:1: ERROR: end of file in argument list 9 File inclusion **************** `m4' allows you to include named files at any point in the input. 9.1 Including named files ========================= There are two builtin macros in `m4' for including files: -- Builtin: include (FILE) -- Builtin: sinclude (FILE) Both macros cause the file named FILE to be read by `m4'. When the end of the file is reached, input is resumed from the previous input file. The expansion of `include' and `sinclude' is therefore the contents of FILE. If FILE does not exist (or cannot be read), the expansion is void, and `include' will fail with an error while `sinclude' is silent. The empty string counts as a file that does not exist. The macros `include' and `sinclude' are recognized only with parameters. include(`none') error-->m4:stdin:1: cannot open `none': No such file or directory => include() error-->m4:stdin:2: cannot open `': No such file or directory => sinclude(`none') => sinclude() => The rest of this section assumes that `m4' is invoked with the `-I' option (*note Invoking m4: Preprocessor features.) pointing to the `m4-1.4.11/examples' directory shipped as part of the GNU `m4' package. The file `m4-1.4.11/examples/incl.m4' in the distribution contains the lines: $ cat examples/incl.m4 =>Include file start =>foo =>Include file end Normally file inclusion is used to insert the contents of a file into the input stream. The contents of the file will be read by `m4' and macro calls in the file will be expanded: $ m4 -I examples define(`foo', `FOO') => include(`incl.m4') =>Include file start =>FOO =>Include file end => The fact that `include' and `sinclude' expand to the contents of the file can be used to define macros that operate on entire files. Here is an example, which defines `bar' to expand to the contents of `incl.m4': $ m4 -I examples define(`bar', include(`incl.m4')) => This is `bar': >>bar<< =>This is bar: >>Include file start =>foo =>Include file end =><< This use of `include' is not trivial, though, as files can contain quotes, commas, and parentheses, which can interfere with the way the `m4' parser works. GNU `m4' seamlessly concatenates the file contents with the next character, even if the included file ended in the middle of a comment, string, or macro call. These conditions are only treated as end of file errors if specified as input files on the command line. In GNU `m4', an alternative method of reading files is using `undivert' (*note Undivert::) on a named file. 9.2 Searching for include files =============================== GNU `m4' allows included files to be found in other directories than the current working directory. If the `--prepend-include' or `-B' command-line option was provided (*note Invoking m4: Preprocessor features.), those directories are searched first, in reverse order that those options were listed on the command line. Then `m4' looks in the current working directory. Next comes the directories specified with the `--include' or `-I' option, in the order found on the command line. Finally, if the `M4PATH' environment variable is set, it is expected to contain a colon-separated list of directories, which will be searched in order. If the automatic search for include-files causes trouble, the `p' debug flag (*note Debug Levels::) can help isolate the problem. 10 Diverting and undiverting output *********************************** Diversions are a way of temporarily saving output. The output of `m4' can at any time be diverted to a temporary file, and be reinserted into the output stream, "undiverted", again at a later time. Numbered diversions are counted from 0 upwards, diversion number 0 being the normal output stream. The number of simultaneous diversions is limited mainly by the memory used to describe them, because GNU `m4' tries to keep diversions in memory. However, there is a limit to the overall memory usable by all diversions taken altogether (512K, currently). When this maximum is about to be exceeded, a temporary file is opened to receive the contents of the biggest diversion still in memory, freeing this memory for other diversions. When creating the temporary file, `m4' honors the value of the environment variable `TMPDIR', and falls back to `/tmp'. So, it is theoretically possible that the number and aggregate size of diversions is limited only by available disk space. Diversions make it possible to generate output in a different order than the input was read. It is possible to implement topological sorting dependencies. For example, GNU Autoconf makes use of diversions under the hood to ensure that the expansion of a prerequisite macro appears in the output prior to the expansion of a dependent macro, regardless of which order the two macros were invoked in the user's input file. 10.1 Diverting output ===================== Output is diverted using `divert': -- Builtin: divert ([NUMBER = `0']) The current diversion is changed to NUMBER. If NUMBER is left out or empty, it is assumed to be zero. If NUMBER cannot be parsed, the diversion is unchanged. The expansion of `divert' is void. When all the `m4' input will have been processed, all existing diversions are automatically undiverted, in numerical order. divert(`1') This text is diverted. divert => This text is not diverted. =>This text is not diverted. ^D => =>This text is diverted. Several calls of `divert' with the same argument do not overwrite the previous diverted text, but append to it. Diversions are printed after any wrapped text is expanded. define(`text', `TEXT') => divert(`1')`diverted text.' divert => m4wrap(`Wrapped text precedes ') => ^D =>Wrapped TEXT precedes diverted text. If output is diverted to a negative diversion, it is simply discarded. This can be used to suppress unwanted output. A common example of unwanted output is the trailing newlines after macro definitions. Here is a common programming idiom in `m4' for avoiding them. divert(`-1') define(`foo', `Macro `foo'.') define(`bar', `Macro `bar'.') divert => Traditional implementations only supported ten diversions. But as a GNU extension, d