Next: , Previous: Shell Functions, Up: Portable Shell


10.12 Limitations of Shell Builtins

No, no, we are serious: some shells do have limitations! :)

You should always keep in mind that any builtin or command may support options, and therefore differ in behavior with arguments starting with a dash. For instance, the innocent `echo "$word"' can give unexpected results when word starts with a dash. It is often possible to avoid this problem using `echo "x$word"', taking the `x' into account later in the pipe.

.
Use . only with regular files (use `test -f'). Bash 2.03, for instance, chokes on `. /dev/null'. Remember that . uses PATH if its argument contains no slashes. Also, some shells, including bash 3.2, implicitly append the current directory to this PATH search, even though Posix forbids it. So if you want to use . on a file foo in the current directory, you must use `. ./foo'.
!
The Unix version 7 shell did not support negating the exit status of commands with !, and this feature is still absent from some shells (e.g., Solaris /bin/sh). Other shells, such as FreeBSD /bin/sh or ash, have bugs when using !:
          $ sh -c '! : | :'; echo $?
          1
          $ ash -c '! : | :'; echo $?
          0
          $ sh -c '! { :; }'; echo $?
          1
          $ ash -c '! { :; }'; echo $?
          {: not found
          Syntax error: "}" unexpected
          2
     

Shell code like this:

          if ! cmp file1 file2 >/dev/null 2>&1; then
            echo files differ or trouble
          fi
     

is therefore not portable in practice. Typically it is easy to rewrite such code, e.g.:

          cmp file1 file2 >/dev/null 2>&1 ||
            echo files differ or trouble
     

More generally, one can always rewrite `! command' as:

          if command; then (exit 1); else :; fi
     

{...}
Bash 3.2 (and earlier versions) sometimes does not properly set `$?' when failing to write redirected output of a compound command. This problem is most commonly observed with `{...}'; it does not occur with `(...)'. For example:
          $ bash -c '{ echo foo; } >/bad; echo $?'
          bash: line 1: /bad: Permission denied
          0
          $ bash -c 'while :; do echo; done >/bad; echo $?'
          bash: line 1: /bad: Permission denied
          0
     

To work around the bug, prepend `:;':

          $ bash -c ':;{ echo foo; } >/bad; echo $?'
          bash: line 1: /bad: Permission denied
          1
     

break
The use of `break 2' etc. is safe.
case
You don't need to quote the argument; no splitting is performed.

You don't need the final `;;', but you should use it.

Posix requires support for case patterns with opening parentheses like this:

          case $file_name in
            (*.c) echo "C source code";;
          esac
     

but the ( in this example is not portable to many Bourne shell implementations, which is a pity for those of us using tools that rely on balanced parentheses. For instance, with Solaris /bin/sh:

          $ case foo in (foo) echo foo;; esac
          error-->syntax error: `(' unexpected
     

The leading `(' can be omitted safely. In contexts where unbalanced parentheses cause other problems, such as when using a case statement as an argument to an Autoconf macro, you can also resort to creative shell comments to supply the balance:

          case $file_name in #(
            *.c) echo "C source code";;
          esac
     

Zsh handles pattern fragments derived from parameter expansions or command substitutions as though quoted:

          $ pat=\?; case aa in ?$pat) echo match;; esac
          $ pat=\?; case a? in ?$pat) echo match;; esac
          match
     

Because of a bug in its fnmatch, Bash fails to properly handle backslashes in character classes:

          bash-2.02$ case /tmp in [/\\]*) echo OK;; esac
          bash-2.02$
     

This is extremely unfortunate, since you are likely to use this code to handle Posix or ms-dos absolute file names. To work around this bug, always put the backslash first:

          bash-2.02$ case '\TMP' in [\\/]*) echo OK;; esac
          OK
          bash-2.02$ case /tmp in [\\/]*) echo OK;; esac
          OK
     

Many Bourne shells cannot handle closing brackets in character classes correctly.

Some shells also have problems with backslash escaping in case you do not want to match the backslash: both a backslash and the escaped character match this pattern. To work around this, specify the character class in a variable, so that quote removal does not apply afterwards, and the special characters don't have to be backslash-escaped:

          $ case '\' in [\<]) echo OK;; esac
          OK
          $ scanset='[<]'; case '\' in $scanset) echo OK;; esac
          $
     

Even with this, Solaris ksh matches a backslash if the set contains any of the characters `|', `&', `(', or `)'.

Conversely, Tru64 ksh (circa 2003) erroneously always matches a closing parenthesis if not specified in a character class:

          $ case foo in *\)*) echo fail ;; esac
          fail
          $ case foo in *')'*) echo fail ;; esac
          fail
     

Some shells, such as Ash 0.3.8, are confused by an empty case/esac:

          ash-0.3.8 $ case foo in esac;
          error-->Syntax error: ";" unexpected (expecting ")")
     

cd
Posix 1003.1-2001 requires that cd must support the -L (“logical”) and -P (“physical”) options, with -L being the default. However, traditional shells do not support these options, and their cd command has the -P behavior.

Portable scripts should assume neither option is supported, and should assume neither behavior is the default. This can be a bit tricky, since the Posix default behavior means that, for example, `ls ..' and `cd ..' may refer to different directories if the current logical directory is a symbolic link. It is safe to use cd dir if dir contains no .. components. Also, Autoconf-generated scripts check for this problem when computing variables like ac_top_srcdir (see Configuration Actions), so it is safe to cd to these variables.

See See Special Shell Variables, for portability problems involving cd and the CDPATH environment variable. Also please see the discussion of the pwd command.

echo
The simple echo is probably the most surprising source of portability troubles. It is not possible to use `echo' portably unless both options and escape sequences are omitted. New applications which are not aiming at portability should use `printf' instead of `echo'.

Don't expect any option. See Preset Output Variables, ECHO_N etc. for a means to simulate -n.

Do not use backslashes in the arguments, as there is no consensus on their handling. For `echo '\n' | wc -l', the sh of Solaris outputs 2, but Bash and Zsh (in sh emulation mode) output 1. The problem is truly echo: all the shells understand `'\n'' as the string composed of a backslash and an `n'.

Because of these problems, do not pass a string containing arbitrary characters to echo. For example, `echo "$foo"' is safe if you know that foo's value cannot contain backslashes and cannot start with `-', but otherwise you should use a here-document like this:

          cat <<EOF
          $foo
          EOF
     

eval
The eval command is useful in limited circumstances, e.g., using commands like `eval table_$key=\$value' and `eval value=table_$key' to simulate a hash table when the key is known to be alphanumeric. However, eval is tricky to use on arbitrary arguments, even when it is implemented correctly.

It is obviously unwise to use `eval $cmd' if the string value of `cmd' was derived from an untrustworthy source. But even if the string value is valid, `eval $cmd' might not work as intended, since it causes field splitting and file name expansion to occur twice, once for the eval and once for the command itself. It is therefore safer to use `eval "$cmd"'. For example, if cmd has the value `cat test?.c', `eval $cmd' might expand to the equivalent of `cat test;.c' if there happens to be a file named test;.c in the current directory; and this in turn mistakenly attempts to invoke cat on the file test and then execute the command .c. To avoid this problem, use `eval "$cmd"' rather than `eval $cmd'.

However, suppose that you want to output the text of the evaluated command just before executing it. Assuming the previous example, `echo "Executing: $cmd"' outputs `Executing: cat test?.c', but this output doesn't show the user that `test;.c' is the actual name of the copied file. Conversely, `eval "echo Executing: $cmd"' works on this example, but it fails with `cmd='cat foo >bar'', since it mistakenly replaces the contents of bar by the string `cat foo'. No simple, general, and portable solution to this problem is known.

You should also be wary of common bugs in eval implementations. In some shell implementations (e.g., older ash, OpenBSD 3.8 sh, pdksh v5.2.14 99/07/13.2, and zsh 4.2.5), the arguments of `eval' are evaluated in a context where `$?' is 0, so they exhibit behavior like this:

          $ false; eval 'echo $?'
          0
     

The correct behavior here is to output a nonzero value, but portable scripts should not rely on this.

You should not rely on LINENO within eval. See Special Shell Variables.

exec
Posix describes several categories of shell built-ins. Special built-ins (such as exit) must impact the environment of the current shell, and need not be available through exec. All other built-ins are regular, and must not propagate variable assignments to the environment of the current shell. However, the group of regular built-ins is further distinguished by commands that do not require a PATH search (such as cd), in contrast to built-ins that are offered as a more efficient version of something that must still be found in a PATH search (such as echo). Posix is not clear on whether exec must work with the list of 17 utilities that are invoked without a PATH search, and many platforms lack an executable for some of those built-ins:
          $ sh -c 'exec cd /tmp'
          sh: line 0: exec: cd: not found
     

All other built-ins that provide utilities specified by Posix must have a counterpart executable that exists on PATH, although Posix allows exec to use the built-in instead of the executable. For example, contrast bash 3.2 and pdksh 5.2.14:

          $ bash -c 'pwd --version' | head -n1
          bash: line 0: pwd: --: invalid option
          pwd: usage: pwd [-LP]
          $ bash -c 'exec pwd --version' | head -n1
          pwd (GNU coreutils) 6.10
          $ pdksh -c 'exec pwd --version' | head -n1
          pdksh: pwd: --: unknown option
     

When it is desired to avoid a regular shell built-in, the workaround is to use some other forwarding command, such as env or nice, that will ensure a path search:

          $ pdksh -c 'exec true --version' | head -n1
          $ pdksh -c 'nice true --version' | head -n1
          true (GNU coreutils) 6.10
          $ pdksh -c 'env true --version' | head -n1
          true (GNU coreutils) 6.10
     

exit
The default value of exit is supposed to be $?; unfortunately, some shells, such as the DJGPP port of Bash 2.04, just perform `exit 0'.
          bash-2.04$ foo=`exit 1` || echo fail
          fail
          bash-2.04$ foo=`(exit 1)` || echo fail
          fail
          bash-2.04$ foo=`(exit 1); exit` || echo fail
          bash-2.04$
     

Using `exit $?' restores the expected behavior.

Some shell scripts, such as those generated by autoconf, use a trap to clean up before exiting. If the last shell command exited with nonzero status, the trap also exits with nonzero status so that the invoker can tell that an error occurred.

Unfortunately, in some shells, such as Solaris /bin/sh, an exit trap ignores the exit command's argument. In these shells, a trap cannot determine whether it was invoked by plain exit or by exit 1. Instead of calling exit directly, use the AC_MSG_ERROR macro that has a workaround for this problem.

export
The builtin export dubs a shell variable environment variable. Each update of exported variables corresponds to an update of the environment variables. Conversely, each environment variable received by the shell when it is launched should be imported as a shell variable marked as exported.

Alas, many shells, such as Solaris /bin/sh, irix 6.3, irix 5.2, AIX 4.1.5, and Digital Unix 4.0, forget to export the environment variables they receive. As a result, two variables coexist: the environment variable and the shell variable. The following code demonstrates this failure:

          #!/bin/sh
          echo $FOO
          FOO=bar
          echo $FOO
          exec /bin/sh $0
     

when run with `FOO=foo' in the environment, these shells print alternately `foo' and `bar', although they should print only `foo' and then a sequence of `bar's.

Therefore you should export again each environment variable that you update; the export can occur before or after the assignment.

Posix is not clear on whether the export of an undefined variable causes the variable to be defined with the value of an empty string, or merely marks any future definition of a variable by that name for export. Various shells behave differently in this regard:

          $ sh -c 'export foo; env | grep foo'
          $ ash -c 'export foo; env | grep foo'
          foo=
     

false
Don't expect false to exit with status 1: in native Solaris /bin/false exits with status 255.
for
To loop over positional arguments, use:
          for arg
          do
            echo "$arg"
          done
     

You may not leave the do on the same line as for, since some shells improperly grok:

          for arg; do
            echo "$arg"
          done
     

If you want to explicitly refer to the positional arguments, given the `$@' bug (see Shell Substitutions), use:

          for arg in ${1+"$@"}; do
            echo "$arg"
          done
     

But keep in mind that Zsh, even in Bourne shell emulation mode, performs word splitting on `${1+"$@"}'; see Shell Substitutions, item `$@', for more.

if
Using `!' is not portable. Instead of:
          if ! cmp -s file file.new; then
            mv file.new file
          fi
     

use:

          if cmp -s file file.new; then :; else
            mv file.new file
          fi
     

There are shells that do not reset the exit status from an if:

          $ if (exit 42); then true; fi; echo $?
          42
     

whereas a proper shell should have printed `0'. This is especially bad in makefiles since it produces false failures. This is why properly written makefiles, such as Automake's, have such hairy constructs:

          if test -f "$file"; then
            install "$file" "$dest"
          else
            :
          fi
     

printf
A format string starting with a `-' can cause problems. Bash interprets it as an option and gives an error. And `--' to mark the end of options is not good in the NetBSD Almquist shell (e.g., 0.4.6) which takes that literally as the format string. Putting the `-' in a `%c' or `%s' is probably easiest:
          printf %s -foo
     

Bash 2.03 mishandles an escape sequence that happens to evaluate to `%':

          $ printf '\045'
          bash: printf: `%': missing format character
     

Large outputs may cause trouble. On Solaris 2.5.1 through 10, for example, /usr/bin/printf is buggy, so when using /bin/sh the command `printf %010000x 123' normally dumps core.

read
Not all shells support -r (Solaris /bin/sh for example).
pwd
With modern shells, plain pwd outputs a “logical” directory name, some of whose components may be symbolic links. These directory names are in contrast to “physical” directory names, whose components are all directories.

Posix 1003.1-2001 requires that pwd must support the -L (“logical”) and -P (“physical”) options, with -L being the default. However, traditional shells do not support these options, and their pwd command has the -P behavior.

Portable scripts should assume neither option is supported, and should assume neither behavior is the default. Also, on many hosts `/bin/pwd' is equivalent to `pwd -P', but Posix does not require this behavior and portable scripts should not rely on it.

Typically it's best to use plain pwd. On modern hosts this outputs logical directory names, which have the following advantages:

Also please see the discussion of the cd command.

set
With the FreeBSD 6.0 shell, the set command (without any options) does not sort its output.

The set builtin faces the usual problem with arguments starting with a dash. Modern shells such as Bash or Zsh understand -- to specify the end of the options (any argument after -- is a parameter, even `-x' for instance), but many traditional shells (e.g., Solaris 10 /bin/sh) simply stop option processing as soon as a non-option argument is found. Therefore, use `dummy' or simply `x' to end the option processing, and use shift to pop it out:

          set x $my_list; shift
     

Avoid `set -', e.g., `set - $my_list'. Posix no longer requires support for this command, and in traditional shells `set - $my_list' resets the -v and -x options, which makes scripts harder to debug.

Some nonstandard shells do not recognize more than one option (e.g., `set -e -x' assigns `-x' to the command line). It is better to combine them:

          set -ex
     

The BSD shell has had several problems with the -e option, partly because BSD make traditionally used -e even though this was incompatible with Posix (see Failure in Make Rules). Older versions of the BSD shell (circa 1990) mishandled `&&', `||', `if', and `case' when -e was in effect, causing the shell to exit unexpectedly in some cases. This was particularly a problem with makefiles, and led to circumlocutions like `sh -c 'test -f file || touch file'', where the seemingly-unnecessary `sh -c '...'' wrapper works around the bug.

Even relatively-recent versions of the BSD shell (e.g., OpenBSD 3.4) wrongly exit with -e if a command within `&&' fails inside a compound statement. For example:

          #! /bin/sh
          set -e
          foo=''
          test -n "$foo" && exit 1
          echo one
          if :; then
            test -n "$foo" && exit 1
          fi
          echo two
     

does not print `two'. One workaround is to use `if test -n "$foo"; then exit 1; fi' rather than `test -n "$foo" && exit 1'. Another possibility is to warn BSD users not to use `sh -e'.

shift
Not only is shifting a bad idea when there is nothing left to shift, but in addition it is not portable: the shell of MIPS RISC/OS 4.52 refuses to do it.

Don't use `shift 2' etc.; it was not in the 7th Edition Bourne shell, and it is also absent in many pre-Posix shells.

source
This command is not portable, as Posix does not require it; use . instead.
test
The test program is the way to perform many file and string tests. It is often invoked by the alternate name `[', but using that name in Autoconf code is asking for trouble since it is an M4 quote character.

The -a, -o, `(', and `)' operands are not portable and should be avoided. Thus, portable uses of test should never have more than four arguments, and scripts should use shell constructs like `&&' and `||' instead. If you combine `&&' and `||' in the same statement, keep in mind that they have equal precedence, so it is often better to parenthesize even when this is redundant. For example:

          # Not portable:
          test "X$a" = "X$b" -a \
            '(' "X$c" != "X$d" -o "X$e" = "X$f" ')'
          
          # Portable:
          test "X$a" = "X$b" &&
            { test "X$c" != "X$d" || test "X$e" = "X$f"; }
     

test does not process options like most other commands do; for example, it does not recognize the -- argument as marking the end of options.

It is safe to use `!' as a test operator. For example, `if test ! -d foo; ...' is portable even though `if ! test -d foo; ...' is not.

test (files)
To enable configure scripts to support cross-compilation, they shouldn't do anything that tests features of the build system instead of the host system. But occasionally you may find it necessary to check whether some arbitrary file exists. To do so, use `test -f' or `test -r'. Do not use `test -x', because 4.3BSD does not have it. Do not use `test -e' either, because Solaris /bin/sh lacks it. To test for symbolic links on systems that have them, use `test -h' rather than `test -L'; either form conforms to Posix 1003.1-2001, but older shells like Solaris 8 /bin/sh support only -h.
test (strings)
Posix says that `test "string"' succeeds if string is not null, but this usage is not portable to traditional platforms like Solaris 10 /bin/sh, which mishandle strings like `!' and `-n'.

Posix also says that `test ! "string"', `test -n "string"' and `test -z "string"' work with any string, but many shells (such as Solaris, AIX 3.2, unicos 10.0.0.6, Digital Unix 4, etc.) get confused if string looks like an operator:

          $ test -n =
          test: argument expected
          $ test ! -n
          test: argument expected
     

Similarly, Posix says that both `test "string1" = "string2"' and `test "string1" != "string2"' work for any pairs of strings, but in practice this is not true for troublesome strings that look like operators or parentheses, or that begin with `-'.

It is best to protect such strings with a leading `X', e.g., `test "Xstring" != X' rather than `test -n "string"' or `test ! "string"'.

It is common to find variations of the following idiom:

          test -n "`echo $ac_feature | sed 's/[-a-zA-Z0-9_]//g'`" &&
            action
     

to take an action when a token matches a given pattern. Such constructs should be avoided by using:

          case $ac_feature in
            *[!-a-zA-Z0-9_]*) action;;
          esac
     

If the pattern is a complicated regular expression that cannot be expressed as a shell pattern, use something like this instead:

          expr "X$ac_feature" : 'X.*[^-a-zA-Z0-9_]' >/dev/null &&
            action
     

`expr "Xfoo" : "Xbar"' is more robust than `echo "Xfoo" | grep "^Xbar"', because it avoids problems when `foo' contains backslashes.

trap
It is safe to trap at least the signals 1, 2, 13, and 15. You can also trap 0, i.e., have the trap run when the script ends (either via an explicit exit, or the end of the script). The trap for 0 should be installed outside of a shell function, or AIX 5.3 /bin/sh will invoke the trap at the end of this function.

Posix says that `trap - 1 2 13 15' resets the traps for the specified signals to their default values, but many common shells (e.g., Solaris /bin/sh) misinterpret this and attempt to execute a “command” named - when the specified conditions arise. There is no portable workaround, except for `trap - 0', for which `trap '' 0' is a portable substitute.

Although Posix is not absolutely clear on this point, it is widely admitted that when entering the trap `$?' should be set to the exit status of the last command run before the trap. The ambiguity can be summarized as: “when the trap is launched by an exit, what is the last command run: that before exit, or exit itself?”

Bash considers exit to be the last command, while Zsh and Solaris /bin/sh consider that when the trap is run it is still in the exit, hence it is the previous exit status that the trap receives:

          $ cat trap.sh
          trap 'echo $?' 0
          (exit 42); exit 0
          $ zsh trap.sh
          42
          $ bash trap.sh
          0
     

The portable solution is then simple: when you want to `exit 42', run `(exit 42); exit 42', the first exit being used to set the exit status to 42 for Zsh, and the second to trigger the trap and pass 42 as exit status for Bash.

The shell in FreeBSD 4.0 has the following bug: `$?' is reset to 0 by empty lines if the code is inside trap.

          $ trap 'false
          
          echo $?' 0
          $ exit
          0
     

Fortunately, this bug only affects trap.

true
Don't worry: as far as we know true is portable. Nevertheless, it's not always a builtin (e.g., Bash 1.x), and the portable shell community tends to prefer using :. This has a funny side effect: when asked whether false is more portable than true Alexandre Oliva answered:
In a sense, yes, because if it doesn't exist, the shell will produce an exit status of failure, which is correct for false, but not for true.

unset
In some nonconforming shells (e.g., Bash 2.05a), unset FOO fails when FOO is not set. Also, Bash 2.01 mishandles unset MAIL in some cases and dumps core.

A few ancient shells lack unset entirely. Nevertheless, because it is extremely useful to disable embarrassing variables such as PS1, you can test for its existence and use it provided you give a neutralizing value when unset is not supported:

          # "|| exit" suppresses any "Segmentation fault" message.
          if ( (MAIL=60; unset MAIL) || exit) >/dev/null 2>&1; then
            unset=unset
          else
            unset=false
          fi
          $unset PS1 || PS1='$ '
     

See Special Shell Variables, for some neutralizing values. Also, see Limitations of Builtins, documentation of export, for the case of environment variables.