8.4.3 Invoking the shell from xargs

Normally, xargs will exec the command you specified directly, without invoking a shell. This is normally the behaviour one would want. It’s somewhat more efficient and avoids problems with shell metacharacters, for example. However, sometimes it is necessary to manipulate the environment of a command before it is run, in a way that xargs does not directly support.

Invoking a shell from xargs is a good way of performing such manipulations. However, some care must be taken to prevent problems, for example unwanted interpretation of shell metacharacters.

This command moves a set of files into an archive directory:

find /foo -maxdepth 1 -atime +366 -exec mv {} /archive \;

However, this will only move one file at a time. We cannot in this case use -exec ... + because the matched file names are added at the end of the command line, while the destination directory would need to be specified last. We also can’t use xargs in the obvious way for the same reason. One way of working around this problem is to make use of the special properties of GNU mv; it has a -t option that allows specifying the target directory before the list of files to be moved. However, while this technique works for GNU mv, it doesn’t solve the more general problem.

Here is a more general technique for solving this problem:

find /foo -maxdepth 1 -atime +366 -print0 |
xargs -r0 sh -c 'mv "$@" /archive' move

Here, a shell is being invoked. There are two shell instances to think about. The first is the shell which launches the xargs command (this might be the shell into which you are typing, for example). The second is the shell launched by xargs (in fact it will probably launch several, one after the other, depending on how many files need to be archived). We’ll refer to this second shell as a subshell.

Our example uses the -c option of sh. Its argument is a shell command to be executed by the subshell. Along with the rest of that command, the $@ is enclosed by single quotes to make sure it is passed to the subshell without being expanded by the parent shell. It is also enclosed with double quotes so that the subshell will expand $@ correctly even if one of the file names contains a space or newline.

The subshell will use any non-option arguments as positional parameters (that is, in the expansion of $@). Because xargs launches the sh -c subshell with a list of files, those files will end up as the expansion of $@.

You may also notice the ‘move’ at the end of the command line. This is used as the value of $0 by the subshell. We include it because otherwise the name of the first file to be moved would be used instead. If that happened it would not be included in the subshell’s expansion of $@, and so it wouldn’t actually get moved.

Another reason to use the sh -c construct could be to perform redirection:

find /usr/include -name '*.h' | xargs grep -wl mode_t |
xargs -r sh -c 'exec emacs "$@" < /dev/tty' Emacs

Notice that we use the shell builtin exec here. That’s simply because the subshell needs to do nothing once Emacs has been invoked. Therefore instead of keeping a sh process around for no reason, we just arrange for the subshell to exec Emacs, saving an extra process creation.

Although GNU xargs and the implementations on some other platforms like BSD support the ‘-o’ option to achieve the same, the above is the portable way to redirect stdin to /dev/tty.

Sometimes, though, it can be helpful to keep the shell process around:

find /foo -maxdepth 1 -atime +366 -print0 |
xargs -r0 sh -c 'mv "$@" /archive || exit 255' move

Here, the shell will exit with status 255 if any mv failed. This causes xargs to stop immediately.