7.5.2 Unambiguous matching ¶

When you want an “inner” or “full” match (discussed previously in Arranging match output), it is important that the matches are unambiguous. Within Gnuastro’s Match program, an ambiguous match is defined when there is more than one “match” within the given aperture or the output depends on the order of the inputs. In other words, to be unambiguous, there should only a single row/entry in both catalogs within the given aperture and the match should not depend on the order of the inputs (which one is given first on the command-line). If there is more than one match in either catalog to the other, Gnuastro’s Match program will remove the match and put all the entries of the second catalog that have multiple matches in a HDU of the output FITS file called FLAGGED-IN-2ND. These are necessary for the match output to be reliable: simply reporting the nearest match when there is more than one match within the given aperture will lead to an ambiguous result.

The issue is best displayed in practice. Let’s assume you have two catalogs called x.fits and o.fits and the points are distributed as follows (in four “clusters”):

$ asttable x.fits
-2    -1
 1     4
 7     0
 8     4
 8    -1
 2     1
 9     0

$ asttable o.fits
-2    -2
 2     3
 4     1
 7     4
 8     0
 9     4

## ASCII visualization of the points in the two tables
|
|            x                o  x  o
|              o
|
|              x   o
|                             x  o  x
|   x                            x
|   o
--------------------------------------->

When you run Gnuastro’s Match program on these two tables (and your aperture is large enough to fit the various “clusters” of points) only the first two points of the two catalogs (on the bottom-left side of the chart) will match. The rest will be flagged and the row-counter of the second input for those matches will be saved in the FLAGGED-IN-2ND HDU of the output. This is very important, especially when you consider the matching aperture to be analogous to your error in the two positions. In other words, while the other points do indeed have a closest match, they have other points that are also within the error/aperture of the match and therefore they will contaminate/pollute any higher-level analysis you want to do with the match.

You can check if your match command produced flagged/ambiguous matches:

• On the lines that Match prints on the terminal, the number of flagged sources will be printed just under the number of unambiguous matches. But this will require human checking (which is not easy in large pipelines!) and is primarily good when you are developing/debugging your higher-level scripts or just understanding your data.
• Read the number of rows of the FLAGGED-IN-2ND HDU of the output and put that in a conditional, like the minimal working example below (you can replace the echo GOOD and echo BAD parts with any series of complex operations):

  $ nflag=$(asttable red-first.fits -hFLAGGED-IN-2ND \
                     --info-num-rows)
  $ if [ $nflag = 0 ]; then echo GOOD; else echo BAD; fi
  

As mentioned above, the FLAGGED-IN-2ND HDU will only contain the row counter (starting from 1) of the ambiguous matches of the second input. To get the full row of the second input from that, you can use the commands below. It uses Table’s Column arithmetic and a second call on Match with the column counters (a one-dimensional match):

$ asttable second.fits --output=2nd-with-counter.fits
           -c'arith $1 counter swap free',_all
$ astmatch original-output.fits  --hdu=FLAGGED-IN-2ND --ccol1=1 \
           2nd-with-counter.fits --hdu2=1             --ccol2=1 \
           --aperture=0.1 --outcols=b_all \
           --output=flagged-in-2nd.fits

The solution to having flagged matches depends on the context, so there is not one easy solution unfortunately. For example one situation where this can occur is the following: You have two catalogs of galaxies and one is from a much deeper image (with a higher spatial resolution) than the other. Due to its depth and higher resolution, the deeper catalog’s points will be much more dense in the same region of the sky and the probability of such ambiguous matches will increase.

To fix the problem above, before doing the match, you can remove all the points in the deep catalog that would not be detectable in the shallow one. To find the points of the deep catalog that should be removed, you can generate the number-counts of your two catalogs (histogram of their magnitudes) and plot them together. This will show your shallower catalog’s completeness limit (the magnitude where the shallower catalog start to deviate (drop) compared to the deeper catalog. You can then remove the sources from the deeper catalog that are fainter than the completeness limit of the shallower catalog (for example, when the limit is on the 24th magnitude, you can use asttable deep.fits --range=MAG,-inf,24). Finally, you can do the match on the the resulting sub-set of the deeper catalog.