GNU Astronomy Utilities


Next: , Previous: , Up: Common options   [Contents][Index]


4.1.2.2 Processing options

Some processing steps are common to several programs, so they are defined as common options to all programs. Note that this class of common options is thus necessarily less common between all the programs than those described in Input/Output options, or Operating mode options options. Also, if they are irrelevant for a program, these options will not display in the --help output of the program.

--minmapsize=INT

The minimum size (in bytes) to store the contents of each main processing array of a program as a file (on the non-volatile HDD/SSD), not in RAM. This can be very useful when you have limited RAM, but need to process large datasets which can be very memory intensive. In such scenarios, without this option, the program will crash.

A random filename is assigned to the array. This file will keep the contents of the array as long as it is necessary and the program will delete it as soon as its not necessary any more.

If the .gnuastro directory exists and is writable, then the random file will be placed in there. Otherwise, the .gnuastro_mmap directory will be checked. If .gnuastro_mmap does not exist, or .gnuastro is not writable, the random file will be directly written in the current directory with the .gnuastro_mmap_ prefix.

By default, the name of the created file, and its size (in bytes) is printed by the program when it is created and later, when its deleted/freed. These messages are useful to the user who has enough RAM, but has forgot to increase the value to --minmapsize (this is often the case). To suppress/disable such messages, use the --quietmmap option.

When this option has a value of 0 (zero, strongly discouraged, see box below), all arrays that use this feature in a program will actually be placed in a file (not in RAM). When this option is larger than all the input datasets, all arrays will be definitely allocated in RAM and the program will run MUCH faster.

Please note that using a non-volatile file (in the HDD/SDD) instead of RAM can significantly increase the program’s running time, especially on HDDs (where read/write is slower). So it is best to give this option large values by default. You can then decrease it for a specific program’s invocation on a large input after you see memory issues arise (for example an error, or the program not aborting and fully consuming your memory).

The random file will be deleted once it is no longer needed by the program. The .gnuastro directory will also be deleted if it has no other contents (you may also have configuration files in this directory, see Configuration files). If you see randomly named files remaining in this directory when the program finishes normally, please send us a bug report so we address the problem, see Report a bug.

Limited number of memory-mapped files: The operating system kernels usually support a limited number of memory-mapped files. Therefore never set --minmapsize to zero or a small number of bytes (so too many files are created). If the kernel capacity is exceeded, the program will crash.

--quietmmap

Don’t print any message when an array is stored in non-volatile memory (HDD/SSD) and not RAM, see the description of --minmapsize (above) for more.

-Z INT[,INT[,...]]
--tilesize=[,INT[,...]]

The size of regular tiles for tessellation, see Tessellation. For each dimension an integer length (in units of data-elements or pixels) is necessary. If the number of input dimensions is different from the number of values given to this option, the program will stop with an error. Values must be separated by commas (,) and can also be fractions (for example 4/2). If they are fractions, the result must be an integer, otherwise an error will be printed.

-M INT[,INT[,...]]
--numchannels=INT[,INT[,...]]

The number of channels for larger input tessellation, see Tessellation. The number and types of acceptable values are similar to --tilesize. The only difference is that instead of length, the integers values given to this option represent the number of channels, not their size.

-F FLT
--remainderfrac=FLT

The fraction of remainder size along all dimensions to add to the first tile. See Tessellation for a complete description. This option is only relevant if --tilesize is not exactly divisible by the input dataset’s size in a dimension. If the remainder size is larger than this fraction (compared to --tilesize), then the remainder size will be added with one regular tile size and divided between two tiles at the start and end of the given dimension.

--workoverch

Ignore the channel borders for the high-level job of the given application. As a result, while the channel borders are respected in defining the small tiles (such that no tile will cross a channel border), the higher-level program operation will ignore them, see Tessellation.

--checktiles

Make a FITS file with the same dimensions as the input but each pixel is replaced with the ID of the tile that it is associated with. Note that the tile IDs start from 0. See Tessellation for more on Tiling an image in Gnuastro.

--oneelempertile

When showing the tile values (for example with --checktiles, or when the program’s output is tessellated) only use one element for each tile. This can be useful when only the relative values given to each tile compared to the rest are important or need to be checked. Since the tiles usually have a large number of pixels within them the output will be much smaller, and so easier to read, write, store, or send.

Note that when the full input size in any dimension is not exactly divisible by the given --tilesize in that dimension, the edge tile(s) will have different sizes (in units of the input’s size), see --remainderfrac. But with this option, all displayed values are going to have the (same) size of one data-element. Hence, in such cases, the image proportions are going to be slightly different with this option.

If your input image is not exactly divisible by the tile size and you want one value per tile for some higher-level processing, all is not lost though. You can see how many pixels were within each tile (for example to weight the values or discard some for later processing) with Gnuastro’s Statistics (see Statistics) as shown below. The output FITS file is going to have two extensions, one with the median calculated on each tile and one with the number of elements that each tile covers. You can then use the where operator in Arithmetic to set the values of all tiles that don’t have the regular area to a blank value.

$ aststatistics --median --number --ontile input.fits    \
                --oneelempertile --output=o.fits
$ REGULAR_AREA=1600    # Check second extension of `o.fits'.
$ astarithmetic o.fits o.fits $REGULAR_AREA ne nan where \
                -h1 -h2

Note that if input.fits also has blank values, then the median on tiles with blank values will also be ignored with the command above (which is desirable).

--inteponlyblank

When values are to be interpolated, only change the values of the blank elements, keep the non-blank elements untouched.

--interpmetric=STR

The metric to use for finding nearest neighbors. Currently it only accepts the Manhattan (or taxicab) metric with manhattan, or the radial metric with radial.

The Manhattan distance between two points is defined with \(|\Delta{x}|+|\Delta{y}|\). Thus the Manhattan metric has the advantage of being fast, but at the expense of being less accurate. The radial distance is the standard definition of distance in a Euclidean space: \(\sqrt{\Delta{x}^2+\Delta{y}^2}\). It is accurate, but the multiplication and square root can slow down the processing.

--interpnumngb=INT

The number of nearby non-blank neighbors to use for interpolation.


Next: , Previous: , Up: Common options   [Contents][Index]