Tables are the high-level products of processing on low-leveler data like images or spectra. For example, in Gnuastro, MakeCatalog will process the pixels over an object and produce a catalog (or table) with the properties of each object such as magnitudes and positions (see MakeCatalog). Each one of these properties is a column in its output catalog (or table) and for each input object, we have a row.
When there are only a small number of objects (rows) and not too many properties (columns), then a simple plain text file is mainly enough to store, transfer, or even use the produced data. However, to be more efficient, astronomers have defined the FITS binary table standard to store data in a binary format (which cannot be seen in a text editor text). This can offer major advantages: the file size will be greatly reduced and the reading and writing will also be faster (because the RAM and CPU also work in binary). The acceptable table formats are fully described in Tables.
Binary tables are not easily readable with basic plain-text editors. There is no fixed/unified standard on how the zero and ones should be interpreted. Unix-like operating systems have flourished because of a simple fact: communication between the various tools is based on human readable characters134. So while the FITS table standards are very beneficial for the tools that recognize them, they are hard to use in the vast majority of available software. This creates limitations for their generic use.
Table is Gnuastro’s solution to this problem.
Table has a large set of operations that you can directly do on any recognized table (such as selecting certain rows and doing arithmetic on the columns).
For operations that Table does not do internally, FITS tables (ASCII or binary) are directly accessible to the users of Unix-like operating systems (in particular those working the command-line or shell, see Command-line interface).
With Table, a FITS table (in binary or ASCII formats) is only one command away from AWK (or any other tool you want to use).
Just like a plain text file that you read with the
You can pipe the output of Table into any other tool for higher-level processing, see the examples in Invoking Table for some simple examples.
In the sections below we describe how to effectively use the Table program. We start with Column arithmetic, where the basic concept and methods of applying arithmetic operations on one or more columns are discussed. Afterwards, in Operation precedence in Table, we review the various types of operations available and their precedence in an instance of calling Table. This is a good place to get a general feeling of all the things you can do with Table. Finally, in Invoking Table, we give some examples and describe each option in Table.
In “The art of Unix programming”, Eric Raymond makes this suggestion to programmers: “When you feel the urge to design a complex binary file format, or a complex binary application protocol, it is generally wise to lie down until the feeling passes.”. This is a great book and strongly recommended, give it a look if you want to truly enjoy your work/life in this environment.