GNU Coreutils ************* This manual documents version 6.9 of the GNU core utilities, including the standard programs for text and file manipulation. Copyright (C) 1994, 1995, 1996, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". 1 Introduction ************** This manual is a work in progress: many sections make no attempt to explain basic concepts in a way suitable for novices. Thus, if you are interested, please get involved in improving this manual. The entire GNU community will benefit. The GNU utilities documented here are mostly compatible with the POSIX standard. Please report bugs to . Remember to include the version number, machine architecture, input files, and any other information needed to reproduce the bug: your input, what you expected, what you got, and why it is wrong. Diffs are welcome, but please include a description of the problem as well, since this is sometimes difficult to infer. *Note Bugs: (gcc)Bugs. This manual was originally derived from the Unix man pages in the distributions, which were written by David MacKenzie and updated by Jim Meyering. What you are reading now is the authoritative documentation for these utilities; the man pages are no longer being maintained. The original `fmt' man page was written by Ross Paterson. Franc,ois Pinard did the initial conversion to Texinfo format. Karl Berry did the indexing, some reorganization, and editing of the results. Brian Youmans of the Free Software Foundation office staff combined the manuals for textutils, fileutils, and sh-utils to produce the present omnibus manual. Richard Stallman contributed his usual invaluable insights to the overall process. 2 Common options **************** Certain options are available in all of these programs. Rather than writing identical descriptions for each of the programs, they are described here. (In fact, every GNU program accepts (or should accept) these options.) Normally options and operands can appear in any order, and programs act as if all the options appear before any operands. For example, `sort -r passwd -t :' acts like `sort -r -t : passwd', since `:' is an option-argument of `-t'. However, if the `POSIXLY_CORRECT' environment variable is set, options must appear before operands, unless otherwise specified for a particular command. A few programs can usefully have trailing operands with leading `-'. With such a program, options must precede operands even if `POSIXLY_CORRECT' is not set, and this fact is noted in the program description. For example, the `env' command's options must appear before its operands, since in some cases the operands specify a command that itself contains options. Some of these programs recognize the `--help' and `--version' options only when one of them is the sole command line argument. `--help' Print a usage message listing all available options, then exit successfully. `--version' Print the version number, then exit successfully. `--' Delimit the option list. Later arguments, if any, are treated as operands even if they begin with `-'. For example, `sort -- -r' reads from the file named `-r'. A single `-' operand is not really an option, though it looks like one. It stands for standard input, or for standard output if that is clear from the context. For example, `sort -' reads from standard input, and is equivalent to plain `sort', and `tee -' writes an extra copy of its input to standard output. Unless otherwise specified, `-' can appear as any operand that requires a file name. 2.1 Exit status =============== Nearly every command invocation yields an integral "exit status" that can be used to change how other commands work. For the vast majority of commands, an exit status of zero indicates success. Failure is indicated by a nonzero value--typically `1', though it may differ on unusual platforms as POSIX requires only that it be nonzero. However, some of the programs documented here do produce other exit status values and a few associate different meanings with the values `0' and `1'. Here are some of the exceptions: `chroot', `env', `expr', `nice', `nohup', `printenv', `sort', `su', `test', `tty'. 2.2 Backup options ================== Some GNU programs (at least `cp', `install', `ln', and `mv') optionally make backups of files before writing new versions. These options control the details of these backups. The options are also briefly mentioned in the descriptions of the particular programs. `-b' `--backup[=METHOD]' Make a backup of each file that would otherwise be overwritten or removed. Without this option, the original versions are destroyed. Use METHOD to determine the type of backups to make. When this option is used but METHOD is not specified, then the value of the `VERSION_CONTROL' environment variable is used. And if `VERSION_CONTROL' is not set, the default backup type is `existing'. Note that the short form of this option, `-b' does not accept any argument. Using `-b' is equivalent to using `--backup=existing'. This option corresponds to the Emacs variable `version-control'; the values for METHOD are the same as those used in Emacs. This option also accepts more descriptive names. The valid METHODs are (unique abbreviations are accepted): `none' `off' Never make backups. `numbered' `t' Always make numbered backups. `existing' `nil' Make numbered backups of files that already have them, simple backups of the others. `simple' `never' Always make simple backups. Please note `never' is not to be confused with `none'. `-S SUFFIX' `--suffix=SUFFIX' Append SUFFIX to each backup file made with `-b'. If this option is not specified, the value of the `SIMPLE_BACKUP_SUFFIX' environment variable is used. And if `SIMPLE_BACKUP_SUFFIX' is not set, the default is `~', just as in Emacs. 2.3 Block size ============== Some GNU programs (at least `df', `du', and `ls') display sizes in "blocks". You can adjust the block size and method of display to make sizes easier to read. The block size used for display is independent of any file system block size. Fractional block counts are rounded up to the nearest integer. The default block size is chosen by examining the following environment variables in turn; the first one that is set determines the block size. `DF_BLOCK_SIZE' This specifies the default block size for the `df' command. Similarly, `DU_BLOCK_SIZE' specifies the default for `du' and `LS_BLOCK_SIZE' for `ls'. `BLOCK_SIZE' This specifies the default block size for all three commands, if the above command-specific environment variables are not set. `BLOCKSIZE' This specifies the default block size for all values that are normally printed as blocks, if neither `BLOCK_SIZE' nor the above command-specific environment variables are set. Unlike the other environment variables, `BLOCKSIZE' does not affect values that are normally printed as byte counts, e.g., the file sizes contained in `ls -l' output. `POSIXLY_CORRECT' If neither `COMMAND_BLOCK_SIZE', nor `BLOCK_SIZE', nor `BLOCKSIZE' is set, but this variable is set, the block size defaults to 512. If none of the above environment variables are set, the block size currently defaults to 1024 bytes in most contexts, but this number may change in the future. For `ls' file sizes, the block size defaults to 1 byte. A block size specification can be a positive integer specifying the number of bytes per block, or it can be `human-readable' or `si' to select a human-readable format. Integers may be followed by suffixes that are upward compatible with the SI prefixes (http://www.bipm.fr/enus/3_SI/si-prefixes.html) for decimal multiples and with the IEC 60027-2 prefixes for binary multiples (http://physics.nist.gov/cuu/Units/binary.html). With human-readable formats, output sizes are followed by a size letter such as `M' for megabytes. `BLOCK_SIZE=human-readable' uses powers of 1024; `M' stands for 1,048,576 bytes. `BLOCK_SIZE=si' is similar, but uses powers of 1000 and appends `B'; `MB' stands for 1,000,000 bytes. A block size specification preceded by `'' causes output sizes to be displayed with thousands separators. The `LC_NUMERIC' locale specifies the thousands separator and grouping. For example, in an American English locale, `--block-size="'1kB"' would cause a size of 1234000 bytes to be displayed as `1,234'. In the default C locale, there is no thousands separator so a leading `'' has no effect. An integer block size can be followed by a suffix to specify a multiple of that size. A bare size letter, or one followed by `iB', specifies a multiple using powers of 1024. A size letter followed by `B' specifies powers of 1000 instead. For example, `1M' and `1MiB' are equivalent to `1048576', whereas `1MB' is equivalent to `1000000'. A plain suffix without a preceding integer acts as if `1' were prepended, except that it causes a size indication to be appended to the output. For example, `--block-size="kB"' displays 3000 as `3kB'. The following suffixes are defined. Large sizes like `1Y' may be rejected by your computer due to limitations of its arithmetic. `kB' kilobyte: 10^3 = 1000. `k' `K' `KiB' kibibyte: 2^10 = 1024. `K' is special: the SI prefix is `k' and the IEC 60027-2 prefix is `Ki', but tradition and POSIX use `k' to mean `KiB'. `MB' megabyte: 10^6 = 1,000,000. `M' `MiB' mebibyte: 2^20 = 1,048,576. `GB' gigabyte: 10^9 = 1,000,000,000. `G' `GiB' gibibyte: 2^30 = 1,073,741,824. `TB' terabyte: 10^12 = 1,000,000,000,000. `T' `TiB' tebibyte: 2^40 = 1,099,511,627,776. `PB' petabyte: 10^15 = 1,000,000,000,000,000. `P' `PiB' pebibyte: 2^50 = 1,125,899,906,842,624. `EB' exabyte: 10^18 = 1,000,000,000,000,000,000. `E' `EiB' exbibyte: 2^60 = 1,152,921,504,606,846,976. `ZB' zettabyte: 10^21 = 1,000,000,000,000,000,000,000 `Z' `ZiB' 2^70 = 1,180,591,620,717,411,303,424. (`Zi' is a GNU extension to IEC 60027-2.) `YB' yottabyte: 10^24 = 1,000,000,000,000,000,000,000,000. `Y' `YiB' 2^80 = 1,208,925,819,614,629,174,706,176. (`Yi' is a GNU extension to IEC 60027-2.) Block size defaults can be overridden by an explicit `--block-size=SIZE' option. The `-k' option is equivalent to `--block-size=1K', which is the default unless the `POSIXLY_CORRECT' environment variable is set. The `-h' or `--human-readable' option is equivalent to `--block-size=human-readable'. The `--si' option is equivalent to `--block-size=si'. 2.4 chown and chgrp: Disambiguating user names and IDs ====================================================== Since the OWNER and GROUP arguments to `chown' and `chgrp' may be specified as names or numeric IDs, there is an apparent ambiguity. What if a user or group _name_ is a string of digits? (1) Should the command interpret it as a user name or as an ID? POSIX requires that `chown' and `chgrp' first attempt to resolve the specified string as a name, and only once that fails, then try to interpret it as an ID. This is troublesome when you want to specify a numeric ID, say 42, and it must work even in a pathological situation where `42' is a user name that maps to some other user ID, say 1000. Simply invoking `chown 42 F', will set `F's owner ID to 1000--not what you intended. GNU `chown' and `chgrp' provide a way to work around this, that at the same time may result in a significant performance improvement by eliminating a database look-up. Simply precede each numeric user ID and/or group ID with a `+', in order to force its interpretation as an integer: chown +42 F chgrp +$numeric_group_id another-file chown +0:+0 / GNU `chown' and `chgrp' skip the name look-up process for each `+'-prefixed string, because a string containing `+' is never a valid user or group name. This syntax is accepted on most common Unix systems, but not on Solaris 10. ---------- Footnotes ---------- (1) Using a number as a user name is common in some environments. 2.5 Sources of random data ========================== The `shuf', `shred', and `sort' commands sometimes need random data to do their work. For example, `sort -R' must choose a hash function at random, and it needs random data to make this selection. Normally these commands use the device file `/dev/urandom' as the source of random data. Typically, this device gathers environmental noise from device drivers and other sources into an entropy pool, and uses the pool to generate random bits. If the pool is short of data, the device reuses the internal pool to produce more bits, using a cryptographically secure pseudorandom number generator. `/dev/urandom' suffices for most practical uses, but applications requiring high-value or long-term protection of private data may require an alternate data source like `/dev/random' or `/dev/arandom'. The set of available sources depends on your operating system. To use such a source, specify the `--random-source=FILE' option, e.g., `shuf --random-source=/dev/random'. The contents of FILE should be as random as possible. An error is reported if FILE does not contain enough bytes to randomize the input adequately. To reproduce the results of an earlier invocation of a command, you can save some random data into a file and then use that file as the random source in earlier and later invocations of the command. Some old-fashioned or stripped-down operating systems lack support for `/dev/urandom'. On these systems commands like `shuf' by default fall back on an internal pseudorandom generator initialized by a small amount of entropy. 2.6 Target directory ==================== The `cp', `install', `ln', and `mv' commands normally treat the last operand specially when it is a directory or a symbolic link to a directory. For example, `cp source dest' is equivalent to `cp source dest/source' if `dest' is a directory. Sometimes this behavior is not exactly what is wanted, so these commands support the following options to allow more fine-grained control: `-T' `--no-target-directory' Do not treat the last operand specially when it is a directory or a symbolic link to a directory. This can help avoid race conditions in programs that operate in a shared area. For example, when the command `mv /tmp/source /tmp/dest' succeeds, there is no guarantee that `/tmp/source' was renamed to `/tmp/dest': it could have been renamed to `/tmp/dest/source' instead, if some other process created `/tmp/dest' as a directory. However, if `mv -T /tmp/source /tmp/dest' succeeds, there is no question that `/tmp/source' was renamed to `/tmp/dest'. In the opposite situation, where you want the last operand to be treated as a directory and want a diagnostic otherwise, you can use the `--target-directory' (`-t') option. `-t DIRECTORY' `--target-directory=DIRECTORY' Use DIRECTORY as the directory component of each destination file name. The interface for most programs is that after processing options and a finite (possibly zero) number of fixed-position arguments, the remaining argument list is either expected to be empty, or is a list of items (usually files) that will all be handled identically. The `xargs' program is designed to work well with this convention. The commands in the `mv'-family are unusual in that they take a variable number of arguments with a special case at the _end_ (namely, the target directory). This makes it nontrivial to perform some operations, e.g., "move all files from here to ../d/", because `mv * ../d/' might exhaust the argument space, and `ls | xargs ...' doesn't have a clean way to specify an extra final argument for each invocation of the subject command. (It can be done by going through a shell command, but that requires more human labor and brain power than it should.) The `--target-directory' (`-t') option allows the `cp', `install', `ln', and `mv' programs to be used conveniently with `xargs'. For example, you can move the files from the current directory to a sibling directory, `d' like this: ls | xargs mv -t ../d -- However, this doesn't move files whose names begin with `.'. If you use the GNU `find' program, you can move those files too, with this command: find . -mindepth 1 -maxdepth 1 \ | xargs mv -t ../d But both of the above approaches fail if there are no files in the current directory, or if any file has a name containing a blank or some other special characters. The following example removes those limitations and requires both GNU `find' and GNU `xargs': find . -mindepth 1 -maxdepth 1 -print0 \ | xargs --null --no-run-if-empty \ mv -t ../d The `--target-directory' (`-t') and `--no-target-directory' (`-T') options cannot be combined. 2.7 Trailing slashes ==================== Some GNU programs (at least `cp' and `mv') allow you to remove any trailing slashes from each SOURCE argument before operating on it. The `--strip-trailing-slashes' option enables this behavior. This is useful when a SOURCE argument may have a trailing slash and specify a symbolic link to a directory. This scenario is in fact rather common because some shells can automatically append a trailing slash when performing file name completion on such symbolic links. Without this option, `mv', for example, (via the system's rename function) must interpret a trailing slash as a request to dereference the symbolic link and so must rename the indirectly referenced _directory_ and not the symbolic link. Although it may seem surprising that such behavior be the default, it is required by POSIX and is consistent with other parts of that standard. 2.8 Traversing symlinks ======================= The following options modify how `chown' and `chgrp' traverse a hierarchy when the `--recursive' (`-R') option is also specified. If more than one of the following options is specified, only the final one takes effect. These options specify whether processing a symbolic link to a directory entails operating on just the symbolic link or on all files in the hierarchy rooted at that directory. These options are independent of `--dereference' and `--no-dereference' (`-h'), which control whether to modify a symlink or its referent. `-H' If `--recursive' (`-R') is specified and a command line argument is a symbolic link to a directory, traverse it. `-L' In a recursive traversal, traverse every symbolic link to a directory that is encountered. `-P' Do not traverse any symbolic links. This is the default if none of `-H', `-L', or `-P' is specified. 2.9 Treating / specially ======================== Certain commands can operate destructively on entire hierarchies. For example, if a user with appropriate privileges mistakenly runs `rm -rf / tmp/junk', that may remove all files on the entire system. Since there are so few legitimate uses for such a command, GNU `rm' normally declines to operate on any directory that resolves to `/'. If you really want to try to remove all the files on your system, you can use the `--no-preserve-root' option, but the default behavior, specified by the `--preserve-option', is safer for most purposes. The commands `chgrp', `chmod' and `chown' can also operate destructively on entire hierarchies, so they too support these options. Although, unlike `rm', they don't actually unlink files, these commands are arguably more dangerous when operating recursively on `/', since they often work much more quickly, and hence damage more files before an alert user can interrupt them. Tradition and POSIX require these commands to operate recursively on `/', so they default to `--no-preserve-root', but using the `--preserve-root' option makes them safer for most purposes. For convenience you can specify `--preserve-root' in an alias or in a shell function. Note that the `--preserve-root' option also ensures that `chgrp' and `chown' do not modify `/' even when dereferencing a symlink pointing to `/'. 2.10 Special built-in utilities =============================== Some programs like `nice' can invoke other programs; for example, the command `nice cat file' invokes the program `cat' by executing the command `cat file'. However, "special built-in utilities" like `exit' cannot be invoked this way. For example, the command `nice exit' does not have a well-defined behavior: it may generate an error message instead of exiting. Here is a list of the special built-in utilities that are standardized by POSIX 1003.1-2004. . : break continue eval exec exit export readonly return set shift times trap unset For example, because `.', `:', and `exec' are special, the commands `nice . foo.sh', `nice :', and `nice exec pwd' do not work as you might expect. Many shells extend this list. For example, Bash has several extra special built-in utilities like `history', and `suspend', and with Bash the command `nice suspend' generates an error message instead of suspending. 2.11 Standards conformance ========================== In a few cases, the GNU utilities' default behavior is incompatible with the POSIX standard. To suppress these incompatibilities, define the `POSIXLY_CORRECT' environment variable. Unless you are checking for POSIX conformance, you probably do not need to define `POSIXLY_CORRECT'. Newer versions of POSIX are occasionally incompatible with older versions. For example, older versions of POSIX required the command `sort +1' to sort based on the second and succeeding fields in each input line, but starting with POSIX 1003.1-2001 the same command is required to sort the file named `+1', and you must instead use the command `sort -k 2' to get the field-based sort. The GNU utilities normally conform to the version of POSIX that is standard for your system. To cause them to conform to a different version of POSIX, define the `_POSIX2_VERSION' environment variable to a value of the form YYYYMM specifying the year and month the standard was adopted. Two values are currently supported for `_POSIX2_VERSION': `199209' stands for POSIX 1003.2-1992, and `200112' stands for POSIX 1003.1-2001. For example, if you have a newer system but are running software that assumes an older version of POSIX and uses `sort +1' or `tail +10', you can work around any compatibility problems by setting `_POSIX2_VERSION=199209' in your environment. 3 Output of entire files ************************ These commands read and write entire files, possibly transforming them in some way. 3.1 `cat': Concatenate and write files ====================================== `cat' copies each FILE (`-' means standard input), or standard input if none are given, to standard output. Synopsis: cat [OPTION] [FILE]... The program accepts the following options. Also see *note Common options::. `-A' `--show-all' Equivalent to `-vET'. `-b' `--number-nonblank' Number all nonblank output lines, starting with 1. `-e' Equivalent to `-vE'. `-E' `--show-ends' Display a `$' after the end of each line. `-n' `--number' Number all output lines, starting with 1. `-s' `--squeeze-blank' Replace multiple adjacent blank lines with a single blank line. `-t' Equivalent to `-vT'. `-T' `--show-tabs' Display TAB characters as `^I'. `-u' Ignored; for POSIX compatibility. `-v' `--show-nonprinting' Display control characters except for LFD and TAB using `^' notation and precede characters that have the high bit set with `M-'. On systems like MS-DOS that distinguish between text and binary files, `cat' normally reads and writes in binary mode. However, `cat' reads in text mode if one of the options `-bensAE' is used or if `cat' is reading from standard input and standard input is a terminal. Similarly, `cat' writes in text mode if one of the options `-bensAE' is used or if standard output is a terminal. An exit status of zero indicates success, and a nonzero value indicates failure. Examples: # Output f's contents, then standard input, then g's contents. cat f - g # Copy standard input to standard output. cat 3.2 `tac': Concatenate and write files in reverse ================================================= `tac' copies each FILE (`-' means standard input), or standard input if none are given, to standard output, reversing the records (lines by default) in each separately. Synopsis: tac [OPTION]... [FILE]... "Records" are separated by instances of a string (newline by default). By default, this separator string is attached to the end of the record that it follows in the file. The program accepts the following options. Also see *note Common options::. `-b' `--before' The separator is attached to the beginning of the record that it precedes in the file. `-r' `--regex' Treat the separator string as a regular expression. Users of `tac' on MS-DOS/MS-Windows should note that, since `tac' reads files in binary mode, each line of a text file might end with a CR/LF pair instead of the Unix-style LF. `-s SEPARATOR' `--separator=SEPARATOR' Use SEPARATOR as the record separator, instead of newline. An exit status of zero indicates success, and a nonzero value indicates failure. 3.3 `nl': Number lines and write files ====================================== `nl' writes each FILE (`-' means standard input), or standard input if none are given, to standard output, with line numbers added to some or all of the lines. Synopsis: nl [OPTION]... [FILE]... `nl' decomposes its input into (logical) pages; by default, the line number is reset to 1 at the top of each logical page. `nl' treats all of the input files as a single document; it does not reset line numbers or logical pages between files. A logical page consists of three sections: header, body, and footer. Any of the sections can be empty. Each can be numbered in a different style from the others. The beginnings of the sections of logical pages are indicated in the input file by a line containing exactly one of these delimiter strings: `\:\:\:' start of header; `\:\:' start of body; `\:' start of footer. The two characters from which these strings are made can be changed from `\' and `:' via options (see below), but the pattern and length of each string cannot be changed. A section delimiter is replaced by an empty line on output. Any text that comes before the first section delimiter string in the input file is considered to be part of a body section, so `nl' treats a file that contains no section delimiters as a single body section. The program accepts the following options. Also see *note Common options::. `-b STYLE' `--body-numbering=STYLE' Select the numbering style for lines in the body section of each logical page. When a line is not numbered, the current line number is not incremented, but the line number separator character is still prepended to the line. The styles are: `a' number all lines, `t' number only nonempty lines (default for body), `n' do not number lines (default for header and footer), `pBRE' number only lines that contain a match for the basic regular expression BRE. *Note Regular Expressions: (grep)Regular Expressions. `-d CD' `--section-delimiter=CD' Set the section delimiter characters to CD; default is `\:'. If only C is given, the second remains `:'. (Remember to protect `\' or other metacharacters from shell expansion with quotes or extra backslashes.) `-f STYLE' `--footer-numbering=STYLE' Analogous to `--body-numbering'. `-h STYLE' `--header-numbering=STYLE' Analogous to `--body-numbering'. `-i NUMBER' `--page-increment=NUMBER' Increment line numbers by NUMBER (default 1). `-l NUMBER' `--join-blank-lines=NUMBER' Consider NUMBER (default 1) consecutive empty lines to be one logical line for numbering, and only number the last one. Where fewer than NUMBER consecutive empty lines occur, do not number them. An empty line is one that contains no characters, not even spaces or tabs. `-n FORMAT' `--number-format=FORMAT' Select the line numbering format (default is `rn'): `ln' left justified, no leading zeros; `rn' right justified, no leading zeros; `rz' right justified, leading zeros. `-p' `--no-renumber' Do not reset the line number at the start of a logical page. `-s STRING' `--number-separator=STRING' Separate the line number from the text line in the output with STRING (default is the TAB character). `-v NUMBER' `--starting-line-number=NUMBER' Set the initial line number on each logical page to NUMBER (default 1). `-w NUMBER' `--number-width=NUMBER' Use NUMBER characters for line numbers (default 6). An exit status of zero indicates success, and a nonzero value indicates failure. 3.4 `od': Write files in octal or other formats =============================================== `od' writes an unambiguous representation of each FILE (`-' means standard input), or standard input if none are given. Synopses: od [OPTION]... [FILE]... od [-abcdfilosx]... [FILE] [[+]OFFSET[.][b]] od [OPTION]... --traditional [FILE] [[+]OFFSET[.][b] [[+]LABEL[.][b]]] Each line of output consists of the offset in the input, followed by groups of data from the file. By default, `od' prints the offset in octal, and each group of file data is a C `short int''s worth of input printed as a single octal number. If OFFSET is given, it specifies how many input bytes to skip before formatting and writing. By default, it is interpreted as an octal number, but the optional trailing decimal point causes it to be interpreted as decimal. If no decimal is specified and the offset begins with `0x' or `0X' it is interpreted as a hexadecimal number. If there is a trailing `b', the number of bytes skipped will be OFFSET multiplied by 512. If a command is of both the first and second forms, the second form is assumed if the last operand begins with `+' or (if there are two operands) a digit. For example, in `od foo 10' and `od +10' the `10' is an offset, whereas in `od 10' the `10' is a file name. The program accepts the following options. Also see *note Common options::. `-A RADIX' `--address-radix=RADIX' Select the base in which file offsets are printed. RADIX can be one of the following: `d' decimal; `o' octal; `x' hexadecimal; `n' none (do not print offsets). The default is octal. `-j BYTES' `--skip-bytes=BYTES' Skip BYTES input bytes before formatting and writing. If BYTES begins with `0x' or `0X', it is interpreted in hexadecimal; otherwise, if it begins with `0', in octal; otherwise, in decimal. Appending `b' multiplies BYTES by 512, `k' by 1024, and `m' by 1048576. `-N BYTES' `--read-bytes=BYTES' Output at most BYTES bytes of the input. Prefixes and suffixes on `bytes' are interpreted as for the `-j' option. `-S N' `--strings[=N]' Instead of the normal output, output only "string constants": at least N consecutive ASCII graphic characters, followed by a null (zero) byte. If N is omitted with `--strings', the default is 3. `-t TYPE' `--format=TYPE' Select the format in which to output the file data. TYPE is a string of one or more of the below type indicator characters. If you include more than one type indicator character in a single TYPE string, or use this option more than once, `od' writes one copy of each output line using each of the data types that you specified, in the order that you specified. Adding a trailing "z" to any type specification appends a display of the ASCII character representation of the printable characters to the output line generated by the type specification. `a' named character, ignoring high-order bit `c' ASCII character or backslash escape, `d' signed decimal `f' floating point `o' octal `u' unsigned decimal `x' hexadecimal The type `a' outputs things like `sp' for space, `nl' for newline, and `nul' for a null (zero) byte. Only the least significant seven bits of each byte is used; the high-order bit is ignored. Type `c' outputs ` ', `\n', and `\0', respectively. Except for types `a' and `c', you can specify the number of bytes to use in interpreting each number in the given data type by following the type indicator character with a decimal integer. Alternately, you can specify the size of one of the C compiler's built-in data types by following the type indicator character with one of the following characters. For integers (`d', `o', `u', `x'): `C' char `S' short `I' int `L' long For floating point (`f'): F float D double L long double `-v' `--output-duplicates' Output consecutive lines that are identical. By default, when two or more consecutive output lines would be identical, `od' outputs only the first line, and puts just an asterisk on the following line to indicate the elision. `-w[N]' `--width[=N]' Dump `n' input bytes per output line. This must be a multiple of the least common multiple of the sizes associated with the specified output types. If this option is not given at all, the default is 16. If N is omitted, the default is 32. The next several options are shorthands for format specifications. GNU `od' accepts any combination of shorthands and format specification options. These options accumulate. `-a' Output as named characters. Equivalent to `-t a'. `-b' Output as octal bytes. Equivalent to `-t o1'. `-c' Output as ASCII characters or backslash escapes. Equivalent to `-t c'. `-d' Output as unsigned decimal two-byte units. Equivalent to `-t u2'. `-f' Output as floats. Equivalent to `-t fF'. `-i' Output as decimal ints. Equivalent to `-t dI'. `-l' Output as decimal long ints. Equivalent to `-t dL'. `-o' Output as octal two-byte units. Equivalent to `-t o2'. `-s' Output as decimal two-byte units. Equivalent to `-t d2'. `-x' Output as hexadecimal two-byte units. Equivalent to `-t x2'. `--traditional' Recognize the non-option label argument that traditional `od' accepted. The following syntax: od --traditional [FILE] [[+]OFFSET[.][b] [[+]LABEL[.][b]]] can be used to specify at most one file and optional arguments specifying an offset and a pseudo-start address, LABEL. The LABEL argument is interpreted just like OFFSET, but it specifies an initial pseudo-address. The pseudo-addresses are displayed in parentheses following any normal address. An exit status of zero indicates success, and a nonzero value indicates failure. 3.5 `base64': Transform data into printable data. ================================================= `base64' transforms data read from a file, or standard input, into (or from) base64 encoded form. The base64 encoded form uses printable ASCII characters to represent binary data, see RFC 3548 (ftp://ftp.rfc-editor.org/in-notes/rfc3548.txt). Synopses: base64 [OPTION]... [FILE] base64 --decode [OPTION]... [FILE] The base64 encoding expands data to roughly 133% of the original. The program accepts the following options. Also see *note Common options::. `-w COLS' `--wrap=COLS' During encoding, wrap lines after COLS characters. This must be a positive number. The default is to wrap after 76 characters. Use the value 0 to disable line wrapping altogether. `-d' `--decode' Change the mode of operation, from the default of encoding data, to decoding data. Input is expected to be base64 encoded data, and the output will be the original data. `-i' `--ignore-garbage' When decoding, newlines are always accepted. During decoding, ignore unrecognized bytes, to permit distorted data to be decoded. An exit status of zero indicates success, and a nonzero value indicates failure. 4 Formatting file contents ************************** These commands reformat the contents of files. 4.1 `fmt': Reformat paragraph text ================================== `fmt' fills and joins lines to produce output lines of (at most) a given number of characters (75 by default). Synopsis: fmt [OPTION]... [FILE]... `fmt' reads from the specified FILE arguments (or standard input if none are given), and writes to standard output. By default, blank lines, spaces between words, and indentation are preserved in the output; successive input lines with different indentation are not joined; tabs are expanded on input and introduced on output. `fmt' prefers breaking lines at the end of a sentence, and tries to avoid line breaks after the first word of a sentence or before the last word of a sentence. A "sentence break" is defined as either the end of a paragraph or a word ending in any of `.?!', followed by two spaces or end of line, ignoring any intervening parentheses or quotes. Like TeX, `fmt' reads entire "paragraphs" before choosing line breaks; the algorithm is a variant of that given by Donald E. Knuth and Michael F. Plass in "Breaking Paragraphs Into Lines", `Software--Practice & Experience' 11, 11 (November 1981), 1119-1184. The program accepts the following options. Also see *note Common options::. `-c' `--crown-margin' "Crown margin" mode: preserve the indentation of the first two lines within a paragraph, and align the left margin of each subsequent line with that of the second line. `-t' `--tagged-paragraph' "Tagged paragraph" mode: like crown margin mode, except that if indentation of the first line of a paragraph is the same as the indentation of the second, the first line is treated as a one-line paragraph. `-s' `--split-only' Split lines only. Do not join short lines to form longer ones. This prevents sample lines of code, and other such "formatted" text from being unduly combined. `-u' `--uniform-spacing' Uniform spacing. Reduce spacing between words to one space, and spacing between sentences to two spaces. `-WIDTH' `-w WIDTH' `--width=WIDTH' Fill output lines up to WIDTH characters (default 75). `fmt' initially tries to make lines about 7% shorter than this, to give it room to balance line lengths. `-p PREFIX' `--prefix=PREFIX' Only lines beginning with PREFIX (possibly preceded by whitespace) are subject to formatting. The prefix and any preceding whitespace are stripped for the formatting and then re-attached to each formatted output line. One use is to format certain kinds of program comments, while leaving the code unchanged. An exit status of zero indicates success, and a nonzero value indicates failure. 4.2 `pr': Paginate or columnate files for printing ================================================== `pr' writes each FILE (`-' means standard input), or standard input if none are given, to standard output, paginating and optionally outputting in multicolumn format; optionally merges all FILEs, printing all in parallel, one per column. Synopsis: pr [OPTION]... [FILE]... By default, a 5-line header is printed at each page: two blank lines; a line with the date, the file name, and the page count; and two more blank lines. A footer of five blank lines is also printed. With the `-F' option, a 3-line header is printed: the leading two blank lines are omitted; no footer is used. The default PAGE_LENGTH in both cases is 66 lines. The default number of text lines changes from 56 (without `-F') to 63 (with `-F'). The text line of the header takes the form `DATE STRING PAGE', with spaces inserted around STRING so that the line takes up the full PAGE_WIDTH. Here, DATE is the date (see the `-D' or `--date-format' option for details), STRING is the centered header string, and PAGE identifies the page number. The `LC_MESSAGES' locale category affects the spelling of PAGE; in the default C locale, it is `Page NUMBER' where NUMBER is the decimal page number. Form feeds in the input cause page breaks in the output. Multiple form feeds produce empty pages. Columns are of equal width, separated by an optional string (default is `space'). For multicolumn output, lines will always be truncated to PAGE_WIDTH (default 72), unless you use the `-J' option. For single column output no line truncation occurs by default. Use `-W' option to truncate lines in that case. The following changes were made in version 1.22i and apply to later versions of `pr': - Brian * Some small LETTER OPTIONS (`-s', `-w') have been redefined for better POSIX compliance. The output of some further cases has been adapted to other Unix systems. These changes are not compatible with earlier versions of the program. * Some NEW CAPITAL LETTER options (`-J', `-S', `-W') have been introduced to turn off unexpected interferences of small letter options. The `-N' option and the second argument LAST_PAGE of `+FIRST_PAGE' offer more flexibility. The detailed handling of form feeds set in the input files requires the `-T' option. * Capital letter options override small letter ones. * Some of the option-arguments (compare `-s', `-e', `-i', `-n') cannot be specified as separate arguments from the preceding option letter (already stated in the POSIX specification). The program accepts the following options. Also see *note Common options::. `+FIRST_PAGE[:LAST_PAGE]' `--pages=FIRST_PAGE[:LAST_PAGE]' Begin printing with page FIRST_PAGE and stop with LAST_PAGE. Missing `:LAST_PAGE' implies end of file. While estimating the number of skipped pages each form feed in the input file results in a new page. Page counting with and without `+FIRST_PAGE' is identical. By default, counting starts with the first page of input file (not first page printed). Line numbering may be altered by `-N' option. `-COLUMN' `--columns=COLUMN' With each single FILE, produce COLUMN columns of output (default is 1) and print columns down, unless `-a' is used. The column width is automatically decreased as COLUMN increases; unless you use the `-W/-w' option to increase PAGE_WIDTH as well. This option might well cause some lines to be truncated. The number of lines in the columns on each page are balanced. The options `-e' and `-i' are on for multiple text-column output. Together with `-J' option column alignment and line truncation is turned off. Lines of full length are joined in a free field format and `-S' option may set field separators. `-COLUMN' may not be used with `-m' option. `-a' `--across' With each single FILE, print columns across rather than down. The `-COLUMN' option must be given with COLUMN greater than one. If a line is too long to fit in a column, it is truncated. `-c' `--show-control-chars' Print control characters using hat notation (e.g., `^G'); print other nonprinting characters in octal backslash notation. By default, nonprinting characters are not changed. `-d' `--double-space' Double space the output. `-D FORMAT' `--date-format=FORMAT' Format header dates using FORMAT, using the same conventions as for the command `date +FORMAT'; *Note date invocation::. Except for directives, which start with `%', characters in FORMAT are printed unchanged. You can use this option to specify an arbitrary string in place of the header date, e.g., `--date-format="Monday morning"'. Normally the date format defaults to `%Y-%m-%d %H:%M' (for example, `2001-12-04 23:59'); but if the `POSIXLY_CORRECT' environment variable is set and the `LC_TIME' locale category specifies the POSIX locale, the default is `%b %e %H:%M %Y' (for example, `Dec 4 23:59 2001'. Time stamps are listed according to the time zone rules specified by the `TZ' environment variable, or by the system default rules if `TZ' is not set. *Note Specifying the Time Zone with `TZ': (libc)TZ Variable. `-e[IN-TABCHAR[IN-TABWIDTH]]' `--expand-tabs[=IN-TABCHAR[IN-TABWIDTH]]' Expand TABs to spaces on input. Optional argument IN-TABCHAR is the input tab character (default is the TAB character). Second optional argument IN-TABWIDTH is the input tab character's width (default is 8). `-f' `-F' `--form-feed' Use a form feed instead of newlines to separate output pages. The default page length of 66 lines is not altered. But the number of lines of text per page changes from default 56 to 63 lines. `-h HEADER' `--header=HEADER' Replace the file name in the header with the centered string HEADER. When using the shell, HEADER should be quoted and should be separated from `-h' by a space. `-i[OUT-TABCHAR[OUT-TABWIDTH]]' `--output-tabs[=OUT-TABCHAR[OUT-TABWIDTH]]' Replace spaces with TABs on output. Optional argument OUT-TABCHAR is the output tab character (default is the TAB character). Second optional argument OUT-TABWIDTH is the output tab character's width (default is 8). `-J' `--join-lines' Merge lines of full length. Used together with the column options `-COLUMN', `-a -COLUMN' or `-m'. Turns off `-W/-w' line truncation; no column alignment used; may be used with `--sep-string[=STRING]'. `-J' has been introduced (together with `-W' and `--sep-string') to disentangle the old (POSIX-compliant) options `-w' and `-s' along with the three column options. `-l PAGE_LENGTH' `--length=PAGE_LENGTH' Set the page length to PAGE_LENGTH (default 66) lines, including the lines of the header [and the footer]. If PAGE_LENGTH is less than or equal to 10 (or <= 3 with `-F'), the header and footer are omitted, and all form feeds set in input files are eliminated, as if the `-T' option had been given. `-m' `--merge' Merge and print all FILEs in parallel, one in each column. If a line is too long to fit in a column, it is truncated, unless the `-J' option is used. `--sep-string[=STRING]' may be used. Empty pages in some FILEs (form feeds set) produce empty columns, still marked by STRING. The result is a continuous line numbering and column marking throughout the whole merged file. Completely empty merged pages show no separators or line numbers. The default header becomes `DATE PAGE' with spaces inserted in the middle; this may be used with the `-h' or `--header' option to fill up the middle blank part. `-n[NUMBER-SEPARATOR[DIGITS]]' `--number-lines[=NUMBER-SEPARATOR[DIGITS]]' Provide DIGITS digit line numbering (default for DIGITS is 5). With multicolumn output the number occupies the first DIGITS column positions of each text column or only each line of `-m' output. With single column output the number precedes each line just as `-m' does. Default counting of the line numbers starts with the first line of the input file (not the first line printed, compare the `--page' option and `-N' option). Optional argument NUMBER-SEPARATOR is the character appended to the line number to separate it from the text followed. The default separator is the TAB character. In a strict sense a TAB is always printed with single column output only. The TAB-width varies with the TAB-position, e.g., with the left MARGIN specified by `-o' option. With multicolumn output priority is given to `equal width of output columns' (a POSIX specification). The TAB-width is fixed to the value of the first column and does not change with different values of left MARGIN. That means a fixed number of spaces is always printed in the place of the NUMBER-SEPARATOR TAB. The tabification depends upon the output position. `-N LINE_NUMBER' `--first-line-number=LINE_NUMBER' Start line counting with the number LINE_NUMBER at first line of first page printed (in most cases not the first line of the input file). `-o MARGIN' `--indent=MARGIN' Indent each line with a margin MARGIN spaces wide (default is zero). The total page width is the size of the margin plus the PAGE_WIDTH set with the `-W/-w' option. A limited overflow may occur with numbered single column output (compare `-n' option). `-r' `--no-file-warnings' Do not print a warning message when an argument FILE cannot be opened. (The exit status will still be nonzero, however.) `-s[CHAR]' `--separator[=CHAR]' Separate columns by a single character CHAR. The default for CHAR is the TAB character without `-w' and `no character' with `-w'. Without `-s' the default separator `space' is set. `-s[char]' turns off line truncation of all three column options (`-COLUMN'|`-a -COLUMN'|`-m') unless `-w' is set. This is a POSIX-compliant formulation. `-SSTRING' `--sep-string[=STRING]' Use STRING to separate output columns. The `-S' option doesn't affect the `-W/-w' option, unlike the `-s' option which does. It does not affect line truncation or column alignment. Without `-S', and with `-J', `pr' uses the default output separator, TAB. Without `-S' or `-J', `pr' uses a `space' (same as `-S" "'). `--sep-string' with no `=STRING' is equivalent to `--sep-string=""'. `-t' `--omit-header' Do not print the usual header [and footer] on each page, and do not fill out the bottom of pages (with blank lines or a form feed). No page structure is produced, but form feeds set in the input files are retained. The predefined pagination is not changed. `-t' or `-T' may be useful together with other options; e.g.: `-t -e4', expand TAB characters in the input file to 4 spaces but don't make any other changes. Use of `-t' overrides `-h'. `-T' `--omit-pagination' Do not print header [and footer]. In addition eliminate all form feeds set in the input files. `-v' `--show-nonprinting' Print nonprinting characters in octal backslash notation. `-w PAGE_WIDTH' `--width=PAGE_WIDTH' Set page width to PAGE_WIDTH characters for multiple text-column output only (default for PAGE_WIDTH is 72). `-s[CHAR]' turns off the default page width and any line truncation and column alignment. Lines of full length are merged, regardless of the column options set. No PAGE_WIDTH setting is possible with single column output. A POSIX-compliant formulation. `-W PAGE_WIDTH' `--page_width=PAGE_WIDTH' Set the page width to PAGE_WIDTH characters. That's valid with and without a column option. Text lines are truncated, unless `-J' is used. Together with one of the three column options (`-COLUMN', `-a -COLUMN' or `-m') column alignment is always used. The separator options `-S' or `-s' don't affect the `-W' option. Default is 72 characters. Without `-W PAGE_WIDTH' and without any of the column options NO line truncation is used (defined to keep downward compatibility and to meet most frequent tasks). That's equivalent to `-W 72 -J'. The header line is never truncated. An exit status of zero indicates success, and a nonzero value indicates failure. 4.3 `fold': Wrap input lines to fit in specified width ====================================================== `fold' writes each FILE (`-' means standard input), or standard input if none are given, to standard output, breaking long lines. Synopsis: fold [OPTION]... [FILE]... By default, `fold' breaks lines wider than 80 columns. The output is split into as many lines as necessary. `fold' counts screen columns by default; thus, a tab may count more than one column, backspace decreases the column count, and carriage return sets the column to zero. The program accepts the following options. Also see *note Common options::. `-b' `--bytes' Count bytes rather than columns, so that tabs, backspaces, and carriage returns are each counted as taking up one column, just like other characters. `-s' `--spaces' Break at word boundaries: the line is broken after the last blank before the maximum line length. If the line contains no such blanks, the line is broken at the maximum line length as usual. `-w WIDTH' `--width=WIDTH' Use a maximum line length of WIDTH columns instead of 80. For compatibility `fold' supports an obsolete option syntax `-WIDTH'. New scripts should use `-w WIDTH' instead. An exit status of zero indicates success, and a nonzero value indicates failure. 5 Output of parts of files ************************** These commands output pieces of the input. 5.1 `head': Output the first part of files ========================================== `head' prints the first part (10 lines by default) of each FILE; it reads from standard input if no files are given or when given a FILE of `-'. Synopsis: head [OPTION]... [FILE]... If more than one FILE is specified, `head' prints a one-line header consisting of: ==> FILE NAME <== before the output for each FILE. The program accepts the following options. Also see *note Common options::. `-c N' `--bytes=N' Print the first N bytes, instead of initial lines. Appending `b' multiplies N by 512, `k' by 1024, and `m' by 1048576. However, if N starts with a `-', print all but the last N bytes of each file. `-n N' `--lines=N' Output the first N lines. However, if N starts with a `-', print all but the last N lines of each file. `-q' `--quiet' `--silent' Never print file name headers. `-v' `--verbose' Always print file name headers. For compatibility `head' also supports an obsolete option syntax `-COUNTOPTIONS', which is recognized only if it is specified first. COUNT is a decimal number optionally followed by a size letter (`b', `k', `m') as in `-c', or `l' to mean count by lines, or other option letters (`cqv'). Scripts intended for standard hosts should use `-c COUNT' or `-n COUNT' instead. If your script must also run on hosts that support only the obsolete syntax, it is usually simpler to avoid `head', e.g., by using `sed 5q' instead of `head -5'. An exit status of zero indicates success, and a nonzero value indicates failure. 5.2 `tail': Output the last part of files ========================================= `tail' prints the last part (10 lines by default) of each FILE; it reads from standard input if no files are given or when given a FILE of `-'. Synopsis: tail [OPTION]... [FILE]... If more than one FILE is specified, `tail' prints a one-line header consisting of: ==> FILE NAME <== before the output for each FILE. GNU `tail' can output any amount of data (some other versions of `tail' cannot). It also has no `-r' option (print in reverse), since reversing a file is really a different job from printing the end of a file; BSD `tail' (which is the one with `-r') can only reverse files that are at most as large as its buffer, which is typically 32 KiB. A more reliable and versatile way to reverse files is the GNU `tac' command. If any option-argument is a number N starting with a `+', `tail' begins printing with the Nth item from the start of each file, instead of from the end. The program accepts the following options. Also see *note Common options::. `-c BYTES' `--bytes=BYTES' Output the last BYTES bytes, instead of final lines. Appending `b' multiplies BYTES by 512, `k' by 1024, and `m' by 1048576. `-f' `--follow[=HOW]' Loop forever trying to read more characters at the end of the file, presumably because the file is growing. If more than one file is given, `tail' prints a header whenever it gets output from a different file, to indicate which file that output is from. There are two ways to specify how you'd like to track files with this option, but that difference is noticeable only when a followed file is removed or renamed. If you'd like to continue to track the end of a growing file even after it has been unlinked, use `--follow=descriptor'. This is the default behavior, but it is not useful if you're tracking a log file that may be rotated (removed or renamed, then reopened). In that case, use `--follow=name' to track the named file by reopening it periodically to see if it has been removed and recreated by some other program. No matter which method you use, if the tracked file is determined to have shrunk, `tail' prints a message saying the file has been truncated and resumes tracking the end of the file from the newly-determined endpoint. When a file is removed, `tail''s behavior depends on whether it is following the name or the descriptor. When following by name, tail can detect that a file has been removed and gives a message to that effect, and if `--retry' has been specified it will continue checking periodically to see if the file reappears. When following a descriptor, tail does not detect that the file has been unlinked or renamed and issues no message; even though the file may no longer be accessible via its original name, it may still be growing. The option values `descriptor' and `name' may be specified only with the long form of the option, not with `-f'. If `POSIXLY_CORRECT' is set, the `-f' option is ignored if no FILE operand is specified and standard input is a FIFO or a pipe. `-F' This option is the same as `--follow=name --retry'. That is, tail will attempt to reopen a file when it is removed. Should this fail, tail will keep trying until it becomes accessible again. `--retry' This option is useful mainly when following by name (i.e., with `--follow=name'). Without this option, when tail encounters a file that doesn't exist or is otherwise inaccessible, it reports that fact and never checks it again. `--sleep-interval=NUMBER' Change the number of seconds to wait between iterations (the default is 1.0). During one iteration, every specified file is checked to see if it has changed size. Historical implementations of `tail' have required that NUMBER be an integer. However, GNU `tail' accepts an arbitrary floating point number (using a period before any fractional digits). `--pid=PID' When following by name or by descriptor, you may specify the process ID, PID, of the sole writer of all FILE arguments. Then, shortly after that process terminates, tail will also terminate. This will work properly only if the writer and the tailing process are running on the same machine. For example, to save the output of a build in a file and to watch the file grow, if you invoke `make' and `tail' like this then the tail process will stop when your build completes. Without this option, you would have had to kill the `tail -f' process yourself. $ make >& makerr & tail --pid=$! -f makerr If you specify a PID that is not in use or that does not correspond to the process that is writing to the tailed files, then `tail' may terminate long before any FILEs stop growing or it may not terminate until long after the real writer has terminated. Note that `--pid' cannot be supported on some systems; `tail' will print a warning if this is the case. `--max-unchanged-stats=N' When tailing a file by name, if there have been N (default n=5) consecutive iterations for which the file has not changed, then `open'/`fstat' the file to determine if that file name is still associated with the same device/inode-number pair as before. When following a log file that is rotated, this is approximately the number of seconds between when tail prints the last pre-rotation lines and when it prints the lines that have accumulated in the new log file. This option is meaningful only when following by name. `-n N' `--lines=N' Output the last N lines. `-q' `--quiet' `--silent' Never print file name headers. `-v' `--verbose' Always print file name headers. For compatibility `tail' also supports an obsolete usage `tail -[COUNT][bcl][f] [FILE]', which is recognized only if it does not conflict with the usage described above. This obsolete form uses exactly one option and at most one file. In the option, COUNT is an optional decimal number optionally followed by a size letter (`b', `c', `l') to mean count by 512-byte blocks, bytes, or lines, optionally followed by `f' which has the same meaning as `-f'. On older systems, the leading `-' can be replaced by `+' in the obsolete option syntax with the same meaning as in counts, and obsolete usage overrides normal usage when the two conflict. This obsolete behavior can be enabled or disabled with the `_POSIX2_VERSION' environment variable (*note Standards conformance::). Scripts intended for use on standard hosts should avoid obsolete syntax and should use `-c COUNT[b]', `-n COUNT', and/or `-f' instead. If your script must also run on hosts that support only the obsolete syntax, you can often rewrite it to avoid problematic usages, e.g., by using `sed -n '$p'' rather than `tail -1'. If that's not possible, the script can use a test like `if tail -c +1 /dev/null 2>&1; then ...' to decide which syntax to use. Even if your script assumes the standard behavior, you should still beware usages whose behaviors differ depending on the POSIX version. For example, avoid `tail - main.c', since it might be interpreted as either `tail main.c' or as `tail -- - main.c'; avoid `tail -c 4', since it might mean either `tail -c4' or `tail -c 10 4'; and avoid `tail +4', since it might mean either `tail ./+4' or `tail -n +4'. An exit status of zero indicates success, and a nonzero value indicates failure. 5.3 `split': Split a file into fixed-size pieces ================================================ `split' creates output files containing consecutive sections of INPUT (standard input if none is given or INPUT is `-'). Synopsis: split [OPTION] [INPUT [PREFIX]] By default, `split' puts 1000 lines of INPUT (or whatever is left over for the last section), into each output file. The output files' names consist of PREFIX (`x' by default) followed by a group of characters (`aa', `ab', ... by default), such that concatenating the output files in traditional sorted order by file name produces the original input file. If the output file names are exhausted, `split' reports an error without deleting the output files that it did create. The program accepts the following options. Also see *note Common options::. `-a LENGTH' `--suffix-length=LENGTH' Use suffixes of length LENGTH. The default LENGTH is 2. `-l LINES' `--lines=LINES' Put LINES lines of INPUT into each output file. For compatibility `split' also supports an obsolete option syntax `-LINES'. New scripts should use `-l LINES' instead. `-b BYTES' `--bytes=BYTES' Put the first BYTES bytes of INPUT into each output file. Appending `b' multiplies BYTES by 512, `k' by 1024, and `m' by 1048576. `-C BYTES' `--line-bytes=BYTES' Put into each output file as many complete lines of INPUT as possible without exceeding BYTES bytes. For lines longer than BYTES bytes, put BYTES bytes into each output file until less than BYTES bytes of the line are left, then continue normally. BYTES has the same format as for the `--bytes' option. `-d' `--numeric-suffixes' Use digits in suffixes rather than lower-case letters. `--verbose' Write a diagnostic to standard error just before each output file is opened. An exit status of zero indicates success, and a nonzero value indicates failure. 5.4 `csplit': Split a file into context-determined pieces ========================================================= `csplit' creates zero or more output files containing sections of INPUT (standard input if INPUT is `-'). Synopsis: csplit [OPTION]... INPUT PATTERN... The contents of the output files are determined by the PATTERN arguments, as detailed below. An error occurs if a PATTERN argument refers to a nonexistent line of the input file (e.g., if no remaining line matches a given regular expression). After every PATTERN has been matched, any remaining input is copied into one last output file. By default, `csplit' prints the number of bytes written to each output file after it has been created. The types of pattern arguments are: `N' Create an output file containing the input up to but not including line N (a positive integer). If followed by a repeat count, also create an output file containing the next N lines of the input file once for each repeat. `/REGEXP/[OFFSET]' Create an output file containing the current line up to (but not including) the next line of the input file that contains a match for REGEXP. The optional OFFSET is an integer. If it is given, the input up to (but not including) the matching line plus or minus OFFSET is put into the output file, and the line after that begins the next section of input. `%REGEXP%[OFFSET]' Like the previous type, except that it does not create an output file, so that section of the input file is effectively ignored. `{REPEAT-COUNT}' Repeat the previous pattern REPEAT-COUNT additional times. The REPEAT-COUNT can either be a positive integer or an asterisk, meaning repeat as many times as necessary until the input is exhausted. The output files' names consist of a prefix (`xx' by default) followed by a suffix. By default, the suffix is an ascending sequence of two-digit decimal numbers from `00' to `99'. In any case, concatenating the output files in sorted order by file name produces the original input file. By default, if `csplit' encounters an error or receives a hangup, interrupt, quit, or terminate signal, it removes any output files that it has created so far before it exits. The program accepts the following options. Also see *note Common options::. `-f PREFIX' `--prefix=PREFIX' Use PREFIX as the output file name prefix. `-b SUFFIX' `--suffix=SUFFIX' Use SUFFIX as the output file name suffix. When this option is specified, the suffix string must include exactly one `printf(3)'-style conversion specification, possibly including format specification flags, a field width, a precision specifications, or all of these kinds of modifiers. The format letter must convert a binary integer argument to readable form; thus, only `d', `i', `u', `o', `x', and `X' conversions are allowed. The entire SUFFIX is given (with the current output file number) to `sprintf(3)' to form the file name suffixes for each of the individual output files in turn. If this option is used, the `--digits' option is ignored. `-n DIGITS' `--digits=DIGITS' Use output file names containing numbers that are DIGITS digits long instead of the default 2. `-k' `--keep-files' Do not remove output files when errors are encountered. `-z' `--elide-empty-files' Suppress the generation of zero-length output files. (In cases where the section delimiters of the input file are supposed to mark the first lines of each of the sections, the first output file will generally be a zero-length file unless you use this option.) The output file sequence numbers always run consecutively starting from 0, even when this option is specified. `-s' `-q' `--silent' `--quiet' Do not print counts of output file sizes. An exit status of zero indicates success, and a nonzero value indicates failure. 6 Summarizing files ******************* These commands generate just a few numbers representing entire contents of files. 6.1 `wc': Print newline, word, and byte counts ============================================== `wc' counts the number of bytes, characters, whitespace-separated words, and newlines in each given FILE, or standard input if none are given or for a FILE of `-'. Synopsis: wc [OPTION]... [FILE]... `wc' prints one line of counts for each file, and if the file was given as an argument, it prints the file name following the counts. If more than one FILE is given, `wc' prints a final line containing the cumulative counts, with the file name `total'. The counts are printed in this order: newlines, words, characters, bytes, maximum line length. Each count is printed right-justified in a field with at least one space between fields so that the numbers and file names normally line up nicely in columns. The width of the count fields varies depending on the inputs, so you should not depend on a particular field width. However, as a GNU extension, if only one count is printed, it is guaranteed to be printed without leading spaces. By default, `wc' prints three counts: the newline, words, and byte counts. Options can specify that only certain counts be printed. Options do not undo others previously given, so wc --bytes --words prints both the byte counts and the word counts. With the `--max-line-length' option, `wc' prints the length of the longest line per file, and if there is more than one file it prints the maximum (not the sum) of those lengths. The program accepts the following options. Also see *note Common options::. `-c' `--bytes' Print only the byte counts. `-m' `--chars' Print only the character counts. `-w' `--words' Print only the word counts. `-l' `--lines' Print only the newline counts. `-L' `--max-line-length' Print only the maximum line lengths. `--files0-from=FILE' Rather than processing files named on the command line, process those named in file FILE; each name is terminated by a null byte. This is useful when the list of file names is so long that it may exceed a command line length limitation. In such cases, running `wc' via `xargs' is undesirable because it splits the list into pieces and makes `wc' print a total for each sublist rather than for the entire list. One way to produce a list of null-byte-terminated file names is with GNU `find', using its `-print0' predicate. For example, to find the length of the longest line in any `.c' or `.h' file in the current hierarchy, do this: find . -name '*.[ch]' -print0 | wc -L --files0-from=- | tail -n1 Do not specify any FILE on the command line when using this option. An exit status of zero indicates success, and a nonzero value indicates failure. 6.2 `sum': Print checksum and block counts ========================================== `sum' computes a 16-bit checksum for each given FILE, or standard input if none are given or for a FILE of `-'. Synopsis: sum [OPTION]... [FILE]... `sum' prints the checksum for each FILE followed by the number of blocks in the file (rounded up). If more than one FILE is given, file names are also printed (by default). (With the `--sysv' option, corresponding file names are printed when there is at least one file argument.) By default, GNU `sum' computes checksums using an algorithm compatible with BSD `sum' and prints file sizes in units of 1024-byte blocks. The program accepts the following options. Also see *note Common options::. `-r' Use the default (BSD compatible) algorithm. This option is included for compatibility with the System V `sum'. Unless `-s' was also given, it has no effect. `-s' `--sysv' Compute checksums using an algorithm compatible with System V `sum''s default, and print file sizes in units of 512-byte blocks. `sum' is provided for compatibility; the `cksum' program (see next section) is preferable in new applications. An exit status of zero indicates success, and a nonzero value indicates failure. 6.3 `cksum': Print CRC checksum and byte counts =============================================== `cksum' computes a cyclic redundancy check (CRC) checksum for each given FILE, or standard input if none are given or for a FILE of `-'. Synopsis: cksum [OPTION]... [FILE]... `cksum' prints the CRC checksum for each file along with the number of bytes in the file, and the file name unless no arguments were given. `cksum' is typically used to ensure that files transferred by unreliable means (e.g., netnews) have not been corrupted, by comparing the `cksum' output for the received files with the `cksum' output for the original files (typically given in the distribution). The CRC algorithm is specified by the POSIX standard. It is not compatible with the BSD or System V `sum' algorithms (see the previous section); it is more robust. The only options are `--help' and `--version'. *Note Common options::. An exit status of zero indicates success, and a nonzero value indicates failure. 6.4 `md5sum': Print or check MD5 digests ======================================== `md5sum' computes a 128-bit checksum (or "fingerprint" or "message-digest") for each specified FILE. Note: The MD5 digest is more reliable than a simple CRC (provided by the `cksum' command) for detecting accidental file corruption, as the chances of accidentally having two files with identical MD5 are vanishingly small. However, it should not be considered truly secure against malicious tampering: although finding a file with a given MD5 fingerprint, or modifying a file so as to retain its MD5 are considered infeasible at the moment, it is known how to produce different files with identical MD5 (a "collision"), something which can be a security issue in certain contexts. For more secure hashes, consider using SHA-1 or SHA-2. *Note sha1sum invocation::, and *note sha2 utilities::. If a FILE is specified as `-' or if no files are given `md5sum' computes the checksum for the standard input. `md5sum' can also determine whether a file and checksum are consistent. Synopsis: md5sum [OPTION]... [FILE]... For each FILE, `md5sum' outputs the MD5 checksum, a flag indicating a binary or text input file, and the file name. If FILE contains a backslash or newline, the line is started with a backslash, and each problematic character in the file name is escaped with a backslash, making the output unambiguous even in the presence of arbitrary file names. If FILE is omitted or specified as `-', standard input is read. The program accepts the following options. Also see *note Common options::. `-b' `--binary' Treat each input file as binary, by reading it in binary mode and outputting a `*' flag. This is the inverse of `--text'. On systems like GNU that do not distinguish between binary and text files, this option merely flags each input file as binary: the MD5 checksum is unaffected. This option is the default on systems like MS-DOS that distinguish between binary and text files, except for reading standard input when standard input is a terminal. `-c' `--check' Read file names and checksum information (not data) from each FILE (or from stdin if no FILE was specified) and report whether the checksums match the contents of the named files. The input to this mode of `md5sum' is usually the output of a prior, checksum-generating run of `md5sum'. Each valid line of input consists of an MD5 checksum, a binary/text flag, and then a file name. Binary files are marked with `*', text with ` '. For each such line, `md5sum' reads the named file and computes its MD5 checksum. Then, if the computed message digest does not match the one on the line with the file name, the file is noted as having failed the test. Otherwise, the file passes the test. By default, for each valid line, one line is written to standard output indicating whether the named file passed the test. After all checks have been performed, if there were any failures, a warning is issued to standard error. Use the `--status' option to inhibit that output. If any listed file cannot be opened or read, if any valid line has an MD5 checksum inconsistent with the associated file, or if no valid line is found, `md5sum' exits with nonzero status. Otherwise, it exits successfully. `--status' This option is useful only when verifying checksums. When verifying checksums, don't generate the default one-line-per-file diagnostic and don't output the warning summarizing any failures. Failures to open or read a file still evoke individual diagnostics to standard error. If all listed files are readable and are consistent with the associated MD5 checksums, exit successfully. Otherwise exit with a status code indicating there was a failure. `-t' `--text' Treat each input file as text, by reading it in text mode and outputting a ` ' flag. This is the inverse of `--binary'. This option is the default on systems like GNU that do not distinguish between binary and text files. On other systems, it is the default for reading standard input when standard input is a terminal. `-w' `--warn' When verifying checksums, warn about improperly formatted MD5 checksum lines. This option is useful only if all but a few lines in the checked input are valid. An exit status of zero indicates success, and a nonzero value indicates failure. 6.5 `sha1sum': Print or check SHA-1 digests =========================================== `sha1sum' computes a 160-bit checksum for each specified FILE. The usage and options of this command are precisely the same as for `md5sum'. *Note md5sum invocation::. Note: The SHA-1 digest is more secure than MD5, and no collisions of it are known (different files having the same fingerprint). However, it is known that they can be produced with considerable, but not unreasonable, resources. For this reason, it is generally considered that SHA-1 should be gradually phased out in favor of the more secure SHA-2 hash algorithms. *Note sha2 utilities::. 6.6 sha2 utilities: Print or check SHA-2 digests ================================================ The commands `sha224sum', `sha256sum', `sha384sum' and `sha512sum' compute checksums of various lengths (respectively 224, 256, 384 and 512 bits), collectively known as the SHA-2 hashes. The usage and options of these commands are precisely the same as for `md5sum'. *Note md5sum invocation::. Note: The SHA384 and SHA512 digests are considerably slower to compute, especially on 32-bit computers, than SHA224 or SHA256. 7 Operating on sorted files *************************** These commands work with (or produce) sorted files. 7.1 `sort': Sort text files =========================== `sort' sorts, merges, or compares all the lines from the given files, or standard input if none are given or for a FILE of `-'. By default, `sort' writes the results to standard output. Synopsis: sort [OPTION]... [FILE]... `sort' has three modes of operation: sort (the default), merge, and check for sortedness. The following options change the operation mode: `-c' `--check' `--check=diagnose-first' Check whether the given file is already sorted: if it is not all sorted, print a diagnostic containing the first out-of-order line and exit with a status of 1. Otherwise, exit successfully. At most one input file can be given. `-C' `--check=quiet' `--check=silent' Exit successfully if the given file is already sorted, and exit with status 1 otherwise. At most one input file can be given. This is like `-c', except it does not print a diagnostic. `-m' `--merge' Merge the given files by sorting them as a group. Each input file must always be individually sorted. It always works to sort instead of merge; merging is provided because it is faster, in the case where it works. A pair of lines is compared as follows: `sort' compares each pair of fields, in the order specified on the command line, according to the associated ordering options, until a difference is found or no fields are left. If no key fields are specified, `sort' uses a default key of the entire line. Finally, as a last resort when all keys compare equal, `sort' compares entire lines as if no ordering options other than `--reverse' (`-r') were specified. The `--stable' (`-s') option disables this "last-resort comparison" so that lines in which all fields compare equal are left in their original relative order. The `--unique' (`-u') option also disables the last-resort comparison. Unless otherwise specified, all comparisons use the character collating sequence specified by the `LC_COLLATE' locale.(1) GNU `sort' (as specified for all GNU utilities) has no limit on input line length or restrictions on bytes allowed within lines. In addition, if the final byte of an input file is not a newline, GNU `sort' silently supplies one. A line's trailing newline is not part of the line for comparison purposes. Exit status: 0 if no error occurred 1 if invoked with `-c' or `-C' and the input is not sorted 2 if an error occurred If the environment variable `TMPDIR' is set, `sort' uses its value as the directory for temporary files instead of `/tmp'. The `--temporary-directory' (`-T') option in turn overrides the environment variable. The following options affect the ordering of output lines. They may be specified globally or as part of a specific key field. If no key fields are specified, global options apply to comparison of entire lines; otherwise the global options are inherited by key fields that do not specify any special options of their own. In pre-POSIX versions of `sort', global options affect only later key fields, so portable shell scripts should specify global options first. `-b' `--ignore-leading-blanks' Ignore leading blanks when finding sort keys in each line. By default a blank is a space or a tab, but the `LC_CTYPE' locale can change this. `-d' `--dictionary-order' Sort in "phone directory" order: ignore all characters except letters, digits and blanks when sorting. By default letters and digits are those of ASCII and a blank is a space or a tab, but the `LC_CTYPE' locale can change this. `-f' `--ignore-case' Fold lowercase characters into the equivalent uppercase characters when comparing so that, for example, `b' and `B' sort as equal. The `LC_CTYPE' locale determines character types. `-g' `--general-numeric-sort' Sort numerically, using the standard C function `strtod' to convert a prefix of each line to a double-precision floating point number. This allows floating point numbers to be specified in scientific notation, like `1.0e-34' and `10e100'. The `LC_NUMERIC' locale determines the decimal-point character. Do not report overflow, underflow, or conversion errors. Use the following collating sequence: * Lines that do not start with numbers (all considered to be equal). * NaNs ("Not a Number" values, in IEEE floating point arithmetic) in a consistent but machine-dependent order. * Minus infinity. * Finite numbers in ascending numeric order (with -0 and +0 equal). * Plus infinity. Use this option only if there is no alternative; it is much slower than `--numeric-sort' (`-n') and it can lose information when converting to floating point. `-i' `--ignore-nonprinting' Ignore nonprinting characters. The `LC_CTYPE' locale determines character types. This option has no effect if the stronger `--dictionary-order' (`-d') option is also given. `-M' `--month-sort' An initial string, consisting of any amount of blanks, followed by a month name abbreviation, is folded to UPPER case and compared in the order `JAN' < `FEB' < ... < `DEC'. Invalid names compare low to valid names. The `LC_TIME' locale category determines the month spellings. By default a blank is a space or a tab, but the `LC_CTYPE' locale can change this. `-n' `--numeric-sort' Sort numerically. The number begins each line and consists of optional blanks, an optional `-' sign, and zero or more digits possibly separated by thousands separators, optionally followed by a decimal-point character and zero or more digits. An empty number is treated as `0'. The `LC_NUMERIC' locale specifies the decimal-point character and thousands separator. By default a blank is a space or a tab, but the `LC_CTYPE' locale can change this. Comparison is exact; there is no rounding error. Neither a leading `+' nor exponential notation is recognized. To compare such strings numerically, use the `--general-numeric-sort' (`-g') option. `-r' `--reverse' Reverse the result of comparison, so that lines with greater key values appear earlier in the output instead of later. `-R' `--random-sort' Sort by hashing the input keys and then sorting the hash values. Choose the hash function at random, ensuring that it is free of collisions so that differing keys have differing hash values. This is like a random permutation of the inputs (*note shuf invocation::), except that keys with the same value sort together. If multiple random sort fields are specified, the same random hash function is used for all fields. To use different random hash functions for different fields, you can invoke `sort' more than once. The choice of hash function is affected by the `--random-source' option. Other options are: `--compress-program=PROG' Compress any temporary files with the program PROG. With no arguments, PROG must compress standard input to standard output, and when given the `-d' option it must decompress standard input to standard output. Terminate with an error if PROG exits with nonzero status. Whitespace and the backslash character should not appear in PROG; they are reserved for future use. `-k POS1[,POS2]' `--key=POS1[,POS2]' Specify a sort field that consists of the part of the line between POS1 and POS2 (or the end of the line, if POS2 is omitted), _inclusive_. Each POS has the form `F[.C][OPTS]', where F is the number of the field to use, and C is the number of the first character from the beginning of the field. Fields and character positions are numbered starting with 1; a character position of zero in POS2 indicates the field's last character. If `.C' is omitted from POS1, it defaults to 1 (the beginning of the field); if omitted from POS2, it defaults to 0 (the end of the field). OPTS are ordering options, allowing individual keys to be sorted according to different rules; see below for details. Keys can span multiple fields. Example: To sort on the second field, use `--key=2,2' (`-k 2,2'). See below for more examples. `-o OUTPUT-FILE' `--output=OUTPUT-FILE' Write output to OUTPUT-FILE instead of standard output. Normally, `sort' reads all input before opening OUTPUT-FILE, so you can safely sort a file in place by using commands like `sort -o F F' and `cat F | sort -o F'. However, `sort' with `--merge' (`-m') can open the output file before reading all input, so a command like `cat F | sort -m -o F - G' is not safe as `sort' might start writing `F' before `cat' is done reading it. On newer systems, `-o' cannot appear after an input file if `POSIXLY_CORRECT' is set, e.g., `sort F -o F'. Portable scripts should specify `-o OUTPUT-FILE' before any input files. `--random-source=FILE' Use FILE as a source of random data used to determine which random hash function to use with the `-R' option. *Note Random sources::. `-s' `--stable' Make `sort' stable by disabling its last-resort comparison. This option has no effect if no fields or global ordering options other than `--reverse' (`-r') are specified. `-S SIZE' `--buffer-size=SIZE' Use a main-memory sort buffer of the given SIZE. By default, SIZE is in units of 1024 bytes. Appending `%' causes SIZE to be interpreted as a percentage of physical memory. Appending `K' multiplies SIZE by 1024 (the default), `M' by 1,048,576, `G' by 1,073,741,824, and so on for `T', `P', `E', `Z', and `Y'. Appending `b' causes SIZE to be interpreted as a byte count, with no multiplication. This option can improve the performance of `sort' by causing it to start with a larger or smaller sort buffer than the default. However, this option affects only the initial buffer size. The buffer grows beyond SIZE if `sort' encounters input lines larger than SIZE. `-t SEPARATOR' `--field-separator=SEPARATOR' Use character SEPARATOR as the field separator when finding the sort keys in each line. By default, fields are separated by the empty string between a non-blank character and a blank character. By default a blank is a space or a tab, but the `LC_CTYPE' locale can change this. That is, given the input line ` foo bar', `sort' breaks it into fields ` foo' and ` bar'. The field separator is not considered to be part of either the field preceding or the field following, so with `sort -t " "' the same input line has three fields: an empty field, `foo', and `bar'. However, fields that extend to the end of the line, as `-k 2', or fields consisting of a range, as `-k 2,3', retain the field separators present between the endpoints of the range. To specify a null character (ASCII NUL) as the field separator, use the two-character string `\0', e.g., `sort -t '\0''. `-T TEMPDIR' `--temporary-directory=TEMPDIR' Use directory TEMPDIR to store temporary files, overriding the `TMPDIR' environment variable. If this option is given more than once, temporary files are stored in all the directories given. If you have a large sort or merge that is I/O-bound, you can often improve performance by using this option to specify directories on different disks and controllers. `-u' `--unique' Normally, output only the first of a sequence of lines that compare equal. For the `--check' (`-c' or `-C') option, check that no pair of consecutive lines compares equal. This option also disables the default last-resort comparison. The commands `sort -u' and `sort | uniq' are equivalent, but this equivalence does not extend to arbitrary `sort' options. For example, `sort -n -u' inspects only the value of the initial numeric string when checking for uniqueness, whereas `sort -n | uniq' inspects the entire line. *Note uniq invocation::. `-z' `--zero-terminated' Treat the input as a set of lines, each terminated by a null character (ASCII NUL) instead of a line feed (ASCII LF). This option can be useful in conjunction with `perl -0' or `find -print0' and `xargs -0' which do the same in order to reliably handle arbitrary file names (even those containing blanks or other special characters). Historical (BSD and System V) implementations of `sort' have differed in their interpretation of some options, particularly `-b', `-f', and `-n'. GNU sort follows the POSIX behavior, which is usually (but not always!) like the System V behavior. According to POSIX, `-n' no longer implies `-b'. For consistency, `-M' has been changed in the same way. This may affect the meaning of character positions in field specifications in obscure cases. The only fix is to add an explicit `-b'. A position in a sort field specified with `-k' may have any of the option letters `Mbdfinr' appended to it, in which case the global ordering options are not used for that particular field. The `-b' option may be independently attached to either or both of the start and end positions of a field specification, and if it is inherited from the global options it will be attached to both. If input lines can contain leading or adjacent blanks and `-t' is not used, then `-k' is typically combined with `-b', `-g', `-M', or `-n'; otherwise the varying numbers of leading blanks in fields can cause confusing results. If the start position in a sort field specifier falls after the end of the line or after the end field, the field is empty. If the `-b' option was specified, the `.C' part of a field specification is counted from the first nonblank character of the field. On older systems, `sort' supports an obsolete origin-zero syntax `+POS1 [-POS2]' for specifying sort keys. This obsolete behavior can be enabled or disabled with the `_POSIX2_VERSION' environment variable (*note Standards conformance::); it can also be enabled when `POSIXLY_CORRECT' is not set by using the obsolete syntax with `-POS2' present. Scripts intended for use on standard hosts should avoid obsolete syntax and should use `-k' instead. For example, avoid `sort +2', since it might be interpreted as either `sort ./+2' or `sort -k 3'. If your script must also run on hosts that support only the obsolete syntax, it can use a test like `if sort -k 1 /dev/null 2>&1; then ...' to decide which syntax to use. Here are some examples to illustrate various combinations of options. * Sort in descending (reverse) numeric order. sort -n -r * Sort alphabetically, omitting the first and second fields and the blanks at the start of the third field. This uses a single key composed of the characters beginning at the start of the first nonblank character in field three and extending to the end of each line. sort -k 3b * Sort numerically on the second field and resolve ties by sorting alphabetically on the third and fourth characters of field five. Use `:' as the field delimiter. sort -t : -k 2,2n -k 5.3,5.4 Note that if you had written `-k 2n' instead of `-k 2,2n' `sort' would have used all characters beginning in the second field and extending to the end of the line as the primary _numeric_ key. For the large majority of applications, treating keys spanning more than one field as numeric will not do what you expect. Also note that the `n' modifier was applied to the field-end specifier for the first key. It would have been equivalent to specify `-k 2n,2' or `-k 2n,2n'. All modifiers except `b' apply to the associated _field_, regardless of whether the modifier character is attached to the field-start and/or the field-end part of the key specifier. * Sort the password file on the fifth field and ignore any leading blanks. Sort lines with equal values in field five on the numeric user ID in field three. Fields are separated by `:'. sort -t : -k 5b,5 -k 3,3n /etc/passwd sort -t : -n -k 5b,5 -k 3,3 /etc/passwd sort -t : -b -k 5,5 -k 3,3n /etc/passwd These three commands have equivalent effect. The first specifies that the first key's start position ignores leading blanks and the second key is sorted numerically. The other two commands rely on global options being inherited by sort keys that lack modifiers. The inheritance works in this case because `-k 5b,5b' and `-k 5b,5' are equivalent, as the location of a field-end lacking a `.C' character position is not affected by whether initial blanks are skipped. * Sort a set of log files, primarily by IPv4 address and secondarily by time stamp. If two lines' primary and secondary keys are identical, output the lines in the same order that they were input. The log files contain lines that look like this: 4.150.156.3 - - [01/Apr/2004:06:31:51 +0000] message 1 211.24.3.231 - - [24/Apr/2004:20:17:39 +0000] message 2 Fields are separated by exactly one space. Sort IPv4 addresses lexicographically, e.g., 212.61.52.2 sorts before 212.129.233.201 because 61 is less than 129. sort -s -t ' ' -k 4.9n -k 4.5M -k 4.2n -k 4.14,4.21 file*.log | sort -s -t '.' -k 1,1n -k 2,2n -k 3,3n -k 4,4n This example cannot be done with a single `sort' invocation, since IPv4 address components are separated by `.' while dates come just after a space. So it is broken down into two invocations of `sort': the first sorts by time stamp and the second by IPv4 address. The time stamp is sorted by year, then month, then day, and finally by hour-minute-second field, using `-k' to isolate each field. Except for hour-minute-second there's no need to specify the end of each key field, since the `n' and `M' modifiers sort based on leading prefixes that cannot cross field boundaries. The IPv4 addresses are sorted lexicographically. The second sort uses `-s' so that ties in the primary key are broken by the secondary key; the first sort uses `-s' so that the combination of the two sorts is stable. * Generate a tags file in case-insensitive sorted order. find src -type f -print0 | sort -z -f | xargs -0 etags --append The use of `-print0', `-z', and `-0' in this case means that file names that contain blanks or other special characters are not broken up by the sort operation. * Shuffle a list of directories, but preserve the order of files within each directory. For instance, one could use this to generate a music playlist in which albums are shuffled but the songs of each album are played in order. ls */* | sort -t / -k 1,1R -k 2,2 ---------- Footnotes ---------- (1) If you use a non-POSIX locale (e.g., by setting `LC_ALL' to `en_US'), then `sort' may produce output that is sorted differently than you're accustomed to. In that case, set the `LC_ALL' environment variable to `C'. Note that setting only `LC_COLLATE' has two problems. First, it is ineffective if `LC_ALL' is also set. Second, it has undefined behavior if `LC_CTYPE' (or `LANG', if `LC_CTYPE' is unset) is set to an incompatible value. For example, you get undefined behavior if `LC_CTYPE' is `ja_JP.PCK' but `LC_COLLATE' is `en_US.UTF-8'. 7.2 `shuf': Shuffling text ========================== `shuf' shuffles its input by outputting a random permutation of its input lines. Each output permutation is equally likely. Synopses: shuf [OPTION]... [FILE] shuf -e [OPTION]... [ARG]... shuf -i LO-HI [OPTION]... `shuf' has three modes of operation that affect where it obtains its input lines. By default, it reads lines from standard input. The following options change the operation mode: `-e' `--echo' Treat each command-line operand as an input line. `-i LO-HI' `--input-range=LO-HI' Act as if input came from a file containing the range of unsigned decimal integers LO...HI, one per line. `shuf''s other options can affect its behavior in all operation modes: `-n LINES' `--head-lines=LINES' Output at most LINES lines. By default, all input lines are output. `-o OUTPUT-FILE' `--output=OUTPUT-FILE' Write output to OUTPUT-FILE instead of standard output. `shuf' reads all input before opening OUTPUT-FILE, so you can safely shuffle a file in place by using commands like `shuf -o F