Appendix F Genfile

This appendix describes genfile, an auxiliary program used in the GNU tar testsuite. If you are not interested in developing GNU tar, skip this appendix.

Initially, genfile was used to generate data files for the testsuite, hence its name. However, new operation modes were being implemented as the testsuite grew more sophisticated, and now genfile is a multi-purpose instrument.

There are four basic operation modes:

File Generation

This is the default mode. In this mode, genfile generates data files.

File Status

In this mode genfile displays status of specified files.

Set File Time

Set last access and modification times of files given in the command line.

Synchronous Execution.

In this mode genfile executes the given program with ‘--checkpoint’ option and executes a set of actions when specified checkpoints are reached.

F.1 Generate Mode

In this mode genfile creates a data file for the test suite. The size of the file is given with the ‘--length’ (‘-l’) option. By default the file contents is written to the standard output, this can be changed using ‘--file’ (‘-f’) command line option. Thus, the following two commands are equivalent:

genfile --length 100 > outfile
genfile --length 100 --file outfile

If ‘--length’ is not given, genfile will generate an empty (zero-length) file.

The command line option ‘--seek=N’ istructs genfile to skip the given number of bytes (N) in the output file before writing to it. It is similar to the ‘seek=N’ of the dd utility.

You can instruct genfile to create several files at one go, by giving it ‘--files-from’ (‘-T’) option followed by a name of file containing a list of file names. Using dash (‘-’) instead of the file name causes genfile to read file list from the standard input. For example:

# Read file names from file ‘file.list’
genfile --files-from file.list
# Read file names from standard input
genfile --files-from -

The list file is supposed to contain one file name per line. To use file lists separated by ASCII NUL character, use ‘--null’ (‘-0’) command line option:

genfile --null --files-from file.list

The default data pattern for filling the generated file consists of first 256 letters of ASCII code, repeated enough times to fill the entire file. This behavior can be changed with ‘--pattern’ option. This option takes a mandatory argument, specifying pattern name to use. Currently two patterns are implemented:

‘--pattern=default’

The default pattern as described above.

‘--pattern=zero’

Fills the file with zeroes.

If no file name was given, the program exits with the code 0. Otherwise, it exits with 0 only if it was able to create a file of the specified length.

Special option ‘--sparse’ (‘-s’) instructs genfile to create a sparse file. Sparse files consist of data fragments, separated by holes or blocks of zeros. On many operating systems, actual disk storage is not allocated for holes, but they are counted in the length of the file. To create a sparse file, genfile should know where to put data fragments, and what data to use to fill them. So, when ‘--sparse’ is given the rest of the command line specifies a so-called file map.

The file map consists of any number of fragment descriptors. Each descriptor is composed of two values: a number, specifying fragment offset from the end of the previous fragment or, for the very first fragment, from the beginning of the file, and contents string, that specifies the pattern to fill the fragment with. File offset can be suffixed with the following quantifiers:

‘k’

‘K’

The number is expressed in kilobytes.

‘m’

‘M’

The number is expressed in megabytes.

‘g’

‘G’

The number is expressed in gigabytes.

Contents string can be either a fragment size or a pattern. Fragment size is a decimal number, prefixed with an equals sign. It can be suffixed with a quantifier, as discussed above. If fragment size is given, the fragment of that size will be filled with the currently selected pattern (see section –pattern) and written to the file.

A pattern is a string of arbitrary ASCII characters. For each of them, genfile will generate a block of data, filled with that character and will write it to the fragment. The size of block is given by ‘--block-size’ option. It defaults to 512. Thus, if pattern consists of n characters, the resulting file fragment will contain n*block-size bytes of data.

The last fragment descriptor can have only file offset part. In this case genfile will create a hole at the end of the file up to the given offset.

A dash appearing as a fragment descriptor instructs genfile to read file map from the standard input. Each line of input should consist of fragment offset and contents string, separated by any amount of whitespace.

For example, consider the following invocation:

genfile --sparse --file sparsefile 0 ABCD 1M EFGHI 2000K

It will create 3101184-bytes long file of the following structure:

The exit code of genfile --sparse command is 0 only if created file is actually sparse. If it is not, the appropriate error message is displayed and the command exists with code 1. The ‘--quite’ (‘-q’) option suppresses this behavior. If ‘--quite’ is given, genfile --sparse exits with code 0 if it was able to create the file, whether the resulting file is sparse or not.

F.2 Status Mode

In status mode, genfile prints file system status for each file specified in the command line. This mode is toggled by ‘--stat’ (‘-S’) command line option. An optional argument to this option specifies output format: a comma-separated list of struct stat fields to be displayed. This list can contain following identifiers:

name

The file name.

dev

st_dev

Device number in decimal.

ino

st_ino

Inode number.

mode[.number]

st_mode[.number]

File mode in octal. Optional number specifies octal mask to be applied to the mode before outputting. For example, --stat mode.777 will preserve lower nine bits of it. Notice, that you can use any punctuation character in place of ‘.’.

nlink

st_nlink

Number of hard links.

uid

st_uid

User ID of owner.

gid

st_gid

Group ID of owner.

size

st_size

File size in decimal.

blksize

st_blksize

The size in bytes of each file block.

blocks

st_blocks

Number of blocks allocated.

atime

st_atime

Time of last access.

mtime

st_mtime

Time of last modification

ctime

st_ctime

Time of last status change

sparse

A boolean value indicating whether the file is ‘sparse’.

Modification times are displayed in UTC as UNIX timestamps, unless suffixed with ‘H’ (for “human-readable”), as in ‘ctimeH’, in which case usual tar tv output format is used.

The default output format is: ‘name,dev,ino,mode, nlink,uid,gid,size,blksize,blocks,atime,mtime,ctime’.

For example, the following command will display file names and corresponding times of last access for each file in the current working directory:

genfile --stat=name,atime *

By default, genfile follows symbolic links and returns information about files pointed to by them. To get information about the symlink files themselves, use the ‘--no-dereference’ (‘-h’) option.

F.3 Set File Time

This mode is requested by the ‘--set-time’ (‘-t’) command line option. In this mode genfile operates similarly to the touch command: for each file listed in the command line, it sets its access and modification times to the current timestamp or to the value given with the ‘--date’ option. The ‘--date’ option takes a date specification in an almost arbitrary format as its argument (see section Date input formats), e.g.:

genfile --set-time --date='2 days ago' a b c

By default, genfile follows symbolic links and sets times of the file they point to. This can be changed by supplying the ‘--no-dereference’ (‘-h’) option: if it is given, genfile will change access and modification times of the symbolic link itself. Notice, that not all operating systems allow this.

F.4 Exec Mode

This mode is designed for testing the behavior of paxutils commands when some of the files change during archiving. It supposes that the command being executed supports ‘--checkpoint’ and ‘--checkpoint-action’ options (see Checkpoints in GNU tar).

The ‘Exec Mode’ is enabled by ‘--run’ command line option (or its alias ‘-r’). The non-optional arguments supply the command line to be executed. Genfile modifies this command line by inserting the following options between the command name and first argument:

--checkpoint=n
--checkpoint-action "echo=genfile checkpoint %u"
--checkpoint-action "wait=SIGUSR1"

Here, n stands for the checkpoint granularity (for GNU tar, it is the number of archive records read or written between each pair of checkpoints). The default value is 1. This value can be changed using the optional argument to the ‘--run’ option. For example, to run actions on each 10th checkpoint:

genfile --run=10 ...

If the command line contains options, it must be preceded by a double-dash (‘--’), which will prevent these options from being interpreted by genfile itself. For example:

genfile --run --checkpoint=2 --truncate foo -- tar -c -f a.tar .

Notice also, that when running tar, its command line may not contain traditional options (cluster of letters without dash).

A set of options is provided for defining checkpoint values and actions to be executed upon reaching them. Checkpoint values are introduced with the ‘--checkpoint’ command line option. Argument to this option is the number of checkpoint in decimal.

Any number of actions may be specified after a checkpoint. Available actions are

‘--cut file’

‘--truncate file’

Truncate file to the size specified by previous ‘--length’ option (or 0, if it is not given).

‘--append file’

Append data to file. The size of data and its pattern are given by previous ‘--length’ and ‘pattern’ options.

‘--touch file’

Update the access and modification times of file. These timestamps are changed to the current time, unless ‘--date’ option was given, in which case they are changed to the specified time. Argument to ‘--date’ option is a date specification in an almost arbitrary format (see section Date input formats).

‘-h’

‘--no-dereference’

Modifies the action of the ‘--touch’ option. If both options are given and file argument to the ‘--touch’ names a symbolic link, genfile will modify access and modification times of the symbolic link file itself, instead the file the symlink points to.

‘--exec command’

Execute given shell command.

‘--delete file’

‘--unlink file’

Delete the named file or directory. If deleting the directory, it must be empty.

Option ‘--verbose’ instructs genfile to print on standard output notifications about checkpoints being executed and to verbosely describe exit status of the command.

While the command is being executed its standard output remains connected to descriptor 1. All messages it prints to file descriptor 2, except checkpoint notifications, are forwarded to standard error.

In exec mode, genfile exits with the exit status of the executed command.

This document was generated on August 23, 2023 using texi2html 5.0.