4.1.5.1 Separate shell variables for multiple outputs ¶

Sometimes your commands print multiple values and you want to use them as different shell variables. Let’s describe the problem (shown in the box below) with an example (that you can reproduce without any external data).

With the commands below, we’ll first make a noisy (\(\sigma=5\)) image (\(100\times100\) pixels) using Arithmetic. Then, we’ll measure¹¹⁷ its mean and standard deviation using Statistics.

$ astarithmetic 100 100 2 makenew 5 mknoise-sigma -oimg.fits

$ aststatistics img.fits --mean --std
-3.10938611484039e-03 4.99607077069093e+00

The first thing that may come to mind is to run Statistics two times, and write the output into separate variables like below:

$ my_std=$(aststatistics img.fits --std)           ## NOT SOLUTION! ##
$ my_mean=$(aststatistics img.fits --mean)         ## NOT SOLUTION! ##

But this is not a good solution because as img.fits becomes larger (more pixels), the time it takes for Statistics to simply load the data into memory can be significant. This will slow down your pipeline and besides wasting your time, it contributes to global warming (by spending energy on an un-necessary action; take this seriously because your pipeline may scale up to involve thousands of large datasets)! Furthermore, besides loading of the input data, Statistics (and Gnuastro in general) is designed to do multiple measurements in one pass over the data as much as possible (to further decrease Gnuastro’s carbon footprint). So when given --mean --std, it will measure both in one pass over the pixels (not two passes!). In other words, in this case, you get the two measurements for the cost of one.

How do you separate the values from the first aststatistics command above? One ugly way is to write the two-number output string into a single shell variable and then separate, or tokenize, the string with two subsequent commands like below:

$ meanstd=$(aststatistics img.fits --mean --std)   ## NOT SOLUTION! ##
$ my_mean=$(echo $meanstd | awk '{print $1}')      ## NOT SOLUTION! ##
$ my_std=$(echo $meanstd | awk '{print $2}')       ## NOT SOLUTION! ##

$ eval "$(aststatistics img.fits --mean --std \
                        | xargs printf "my_mean=%s; my_std=%s")"

Let’s review the solution (in more detail):

1. We pipe the output into xargs¹¹⁸ (extended arguments) which puts the two numbers it gets from the pipe, as arguments for printf (formatted print; because printf doesn’t take input from pipes).
2. Within the printf call, we write the values after putting a variable name and equal-sign, and in between them we put a ; (as if it was a shell command). The %s tells printf to print each input as a string (not to interpret it as a number and loose precision). Here is the output of this phase:

   $ aststatistics img.fits --mean --std \
                         | xargs printf "my_mean=%s; my_std=%s"
   my_mean=-3.10938611484039e-03; my_std=4.99607077069093e+00
   

3. But the output above is a string! To evaluate this string as a command, we give it to the eval command like above.

After the solution above, you will have the two my_mean and my_std variables to use separately in your pipeline:

$ echo $my_mean
-3.10938611484039e-03
$ echo $my_std
4.99607077069093e+00

This eval-based solution has been tested in in GNU Bash, Dash and Zsh and it works nicely in them (is “portable”). This is because the constructs used here are pretty low-level (and widely available).

For examples usages of this technique, see the following sections: Extracting a single spectrum and plotting it and Pseudo narrow-band images.