GNU Astronomy Utilities


Next: , Previous: , Up: MakeCatalog   [Contents][Index]


7.3.2 Quantifying measurement limits

No measurement on a real dataset can be perfect: you can only reach a certain level/limit of accuracy. Therefore, a meaningful (scientific) analysis requires an understanding of these limits for the dataset and your analysis tools: different datasets (images in the case of MakeCatalog) have different noise properties and different detection methods (one method/algorith/software that is run with a different set of parameters is considered as a different detection method) will have different abilities to detect or measure certain kinds of signal (astronomical objects) and their properties in an image. Hence, quantifying the detection and measurement limitations with a particular dataset and analysis tool is the most crucial/critical aspect of any high-level analysis.

In this section we discuss some of the most general limits that are very important in any astronomical data analysis and how MakeCatalog makes it easy to find them. Depending on the higher-level analysis, there are more tests that must be done, but these are usually necessary in any case. In astronomy, it is common to use the magnitude (a unit-less scale) and physical units, see Flux Brightness and magnitude. Therefore all the measurements discussed here are defined in units of magnitudes.

Surface brightness limit (of whole dataset)

As we make more observations on one region of the sky, and add the observations into one dataset, we are able to decrease the standard deviation of the noise in each pixel99. Qualitatively, this decrease manifests its self by making fainter (per pixel) parts of the objects in the image more visible. Technically, this is known as surface brightness. Quantitatively, it increases the Signal to noise ratio, since the signal increases faster than noise with more data. It is very important to have in mind that here, noise is defined per pixel (or in the units of our data measurement), not per object.

You can think of the noise as muddy water that is completely covering a flat ground100 with some regions higher than the others101 in it. In this analogy, height (from the ground) is surface brightness. Let’s assume that in your first observation the muddy water has just been stirred and you can’t see anything through it. As you wait and make more observations, the mud settles down and the depth of the transparent water increases, making the summits of hills visible. As the depth of clear water increases, the parts of the hills with lower heights (less parts with lower surface brightness) can be seen more clearly.

The outputs of NoiseChisel include the Sky standard deviation (\(\sigma\)) on every group of pixels (a mesh) that were calculated from the undetected pixels in that mesh, see Tessellation and NoiseChisel output. Let’s take \(\sigma_m\) as the median \(\sigma\) over the successful meshes in the image (prior to interpolation or smoothing).

On different instruments pixels have different physical sizes (for example in micro-meters, or spatial angle over the sky), nevertheless, a pixel is our unit of data collection. In other words, while quantifying the noise, the physical or projected size of the pixels is irrelevant. We thus define the Surface brightness limit or depth, in units of magnitude/pixel, of a data-set, with zeropoint magnitude \(z\), with the \(n\)th multiple of \(\sigma_m\) as (see Flux Brightness and magnitude):

$$SB_{\rm Pixel}=-2.5\times\log_{10}{(n\sigma_m)}+z$$

As an example, the XDF survey covers part of the sky that the Hubble space telescope has observed the most (for 85 orbits) and is consequently very small (\(\sim4\) arcmin\(^2\)). On the other hand, the CANDELS survey, is one of the widest multi-color surveys covering several fields (about 720 arcmin\(^2\)) but its deepest fields have only 9 orbits observation. The depth of the XDF and CANDELS-deep surveys in the near infrared WFC3/F160W filter are respectively 34.40 and 32.45 magnitudes/pixel. In a single orbit image, this same field has a depth of 31.32. Recall that a larger magnitude corresponds to less brightness.

The low-level magnitude/pixel measurement above is only useful when all the datasets you want to use belong to one instrument (telescope and camera). However, you will often find yourself using datasets from various instruments with different pixel scales (projected pixel sizes). If we know the pixel scale, we can obtain a more easily comparable surface brightness limit in units of: magnitude/arcsec\(^2\). Let’s assume that the dataset has a zeropoint value of \(z\), and every pixel is \(p\) arcsec\(^2\) (so \(A/p\) is the number of pixels that cover an area of \(A\) arcsec\(^2\)). If the \(n\)th multiple of \(\sigma_m\) is desired, then the surface brightness (in units of magnitudes per A arcsec\(^2\)) is102:

$$SB_{\rm Projected}=-2.5\times\log_{10}{\left(n\sigma_m\sqrt{A\over p}\right)+z}$$

Note that this is an extrapolation of the actually measured value of \(\sigma_m\) (which was per pixel). So it should be used with extreme care (for example the dataset must have an approximately flat depth). For each detection over the dataset, you can estimate an upper-limit magnitude which actually uses the detection’s area/footprint. It doesn’t extrapolate and even accounts for correlated noise features. Therefore, the upper-limit magnitude is a much better measure of your dataset’s surface brightness limit for each particular object.

MakeCatalog will calculate the input dataset’s \(SB_{\rm Pixel}\) and \(SB_{\rm Projected}\) and write them as comments/meta-data in the output catalog(s). Just note that \(SB_{\rm Projected}\) is only calculated if the input has World Coordinate System (WCS).

Completeness limit (of each detection)

As the surface brightness of the objects decreases, the ability to detect them will also decrease. An important statistic is thus the fraction of objects of similar morphology and brightness that will be identified with our detection algorithm/parameters in the given image. This fraction is known as completeness. For brighter objects, completeness is 1: all bright objects that might exist over the image will be detected. However, as we go to lower surface brightness objects, we fail to detect some and gradually we are not able to detect anything any more. For a given profile, the magnitude where the completeness drops below a certain level usually above \(90\%\) is known as the completeness limit.

Another important parameter in measuring completeness is purity: the fraction of true detections to all true detections. In effect purity is the measure of contamination by false detections: the higher the purity, the lower the contamination. Completeness and purity are anti-correlated: if we can allow a large number of false detections (that we might be able to remove by other means), we can significantly increase the completeness limit.

One traditional way to measure the completeness and purity of a given sample is by embedding mock profiles in regions of the image with no detection. However in such a study we must be really careful to choose model profiles as similar to the target of interest as possible.

Magnitude measurement error (of each detection)

Any measurement has an error and this includes the derived magnitude for an object. Note that this value is only meaningful when the object’s magnitude is brighter than the upper-limit magnitude (see the next items in this list). As discussed in Flux Brightness and magnitude, the magnitude (\(M\)) of an object with brightness \(B\) and Zeropoint magnitude \(z\) can be written as:

$$M=-2.5\log_{10}(B)+z$$

Calculating the derivative with respect to \(B\), we get:

$${dM\over dB} = {-2.5\over {B\times ln(10)}}$$

From the Tailor series (\(\Delta{M}=dM/dB\times\Delta{B}\)), we can write:

$$\Delta{M} = \left|{-2.5\over ln(10)}\right|\times{\Delta{B}\over{B}}$$

But, \(\Delta{B}/B\) is just the inverse of the Signal-to-noise ratio (\(S/N\)), so we can write the error in magnitude in terms of the signal-to-noise ratio:

$$\Delta{M} = {2.5\over{S/N\times ln(10)}} $$

MakeCatalog uses this relation to estimate the magnitude errors. The signal-to-noise ratio is calculated in different ways for clumps and objects (see Akhlaghi and Ichikawa [2015]), but this single equation can be used to estimate the measured magnitude error afterwards for any type of target.

Upper limit magnitude (of each detection)

Due to the noisy nature of data, it is possible to get arbitrarily low values for a faint object’s brightness (or arbitrarily high magnitudes). Given the scatter caused by the noise, such small values are meaningless: another similar depth observation will give a radically different value. This problem is most common when you use one image/filter to generate target labels (which specify which pixels belong to which object, see NoiseChisel output and MakeCatalog) and another image/filter to generate a catalog for measuring colors.

The object might not be visible in the filter used for the latter image, or the image depth (see above) might be much shallower. So you will get unreasonably faint magnitudes. For example when the depth of the image is 32 magnitudes, a measurement that gives a magnitude of 36 for a \(\sim100\) pixel object is clearly unreliable. In another similar depth image, we might measure a magnitude of 30 for it, and yet another might give 33. Furthermore, due to the noise scatter so close to the depth of the data-set, the total brightness might actually get measured as a negative value, so no magnitude can be defined (recall that a magnitude is a base-10 logarithm).

Using such unreliable measurements will directly affect our analysis, so we must not use them. However, all is not lost! Given our limited depth, there is one thing we can deduce about the object’s magnitude: we can say that if something actually exists here (possibly buried deep under the noise), it must have a magnitude that is fainter than an upper limit magnitude. To find this upper limit magnitude, we place the object’s footprint (segmentation map) over random parts of the image where there are no detections, so we only have pure (possibly correlated) noise and undetected objects. Doing this a large number of times will give us a distribution of brightness values. The standard deviation (\(\sigma\)) of that distribution can be used to quantify the upper limit magnitude.

Traditionally, faint/small object photometry was done using fixed circular apertures (for example with a diameter of \(N\) arc-seconds). In this way, the upper limit was like the depth discussed above: one value for the whole image. But with the much more advanced hardware and software of today, we can make customized segmentation maps for each object. The number of pixels (are of the object) used directly affects the final distribution and thus magnitude. Also the image correlated noise might actually create certain patters, so the shape of the object can also affect the result. So in MakeCatalog, the upper limit magnitude is found for each object in the image separately. Not one value for the whole image.


Footnotes

(99)

This is true for any noisy data, not just astronomical images

(100)

The ground is the sky value in this analogy, see Sky value. Note that this analogy only holds for a flat sky value across the surface of the image or ground.

(101)

The peaks are the brightest parts of astronomical objects in this analogy.

(102)

If we have \(N\) datasets, each with noise \(\sigma\), the noise of a combined dataset will increase as \(\sqrt{N}\sigma\).


Next: , Previous: , Up: MakeCatalog   [Contents][Index]