Numerical programming is an extensive area; if you need to develop
sophisticated numerical algorithms then `gawk` may not be
the ideal tool, and this documentation may not be sufficient.
It might require digesting a book or two^{1}
to really internalize how to compute
with ideal accuracy and precision,
and the result often depends on the particular application.

NOTE:A floating-point calculation's accuracy is how close it comes to the real value. This is as opposed to the precision, which usually refers to the number of bits used to represent the number (see the Wikipedia article for more information).

There are two options for doing floating-point calculations:
hardware floating-point (as used by standard `awk` and
the default for `gawk`), and arbitrary-precision
floating-point, which is software based.
From this point forward, this chapter
aims to provide enough information to understand both, and then
will focus on `gawk`'s facilities for the latter.^{2}

Binary floating-point representations and arithmetic are inexact. Simple values like 0.1 cannot be precisely represented using binary floating-point numbers, and the limited precision of floating-point numbers means that slight changes in the order of operations or the precision of intermediate storage can change the result. To make matters worse, with arbitrary precision floating-point, you can set the precision before starting a computation, but then you cannot be sure of the number of significant decimal places in the final result.

Sometimes, before you start to write any code, you should think more about what you really want and what's really happening. Consider the two numbers in the following example:

x = 0.875 # 1/2 + 1/4 + 1/8 y = 0.425

Unlike the number in `y`

, the number stored in `x`

is exactly representable
in binary since it can be written as a finite sum of one or
more fractions whose denominators are all powers of two.
When `gawk` reads a floating-point number from
program source, it automatically rounds that number to whatever
precision your machine supports. If you try to print the numeric
content of a variable using an output format string of `"%.17g"`

,
it may not produce the same number as you assigned to it:

$gawk 'BEGIN { x = 0.875; y = 0.425>printf("%0.17g, %0.17g\n", x, y) }'-| 0.875, 0.42499999999999999

Often the error is so small you do not even notice it, and if you do,
you can always specify how much precision you would like in your output.
Usually this is a format string like `"%.15g"`

, which when
used in the previous example, produces an output identical to the input.

Because the underlying representation can be a little bit off from the exact value, comparing floating-point values to see if they are equal is generally not a good idea. Here is an example where it does not work like you expect:

$gawk 'BEGIN { print (0.1 + 12.2 == 12.3) }'-| 0

The loss of accuracy during a single computation with floating-point numbers
usually isn't enough to worry about. However, if you compute a value
which is the result of a sequence of floating point operations,
the error can accumulate and greatly affect the computation itself.
Here is an attempt to compute the value of the constant
*pi* using one of its many series representations:

BEGIN { x = 1.0 / sqrt(3.0) n = 6 for (i = 1; i < 30; i++) { n = n * 2.0 x = (sqrt(x * x + 1) - 1) / x printf("%.15f\n", n * x) } }

When run, the early errors propagating through later computations cause the loop to terminate prematurely after an attempt to divide by zero.

$gawk -f pi.awk-| 3.215390309173475 -| 3.159659942097510 -| 3.146086215131467 -| 3.142714599645573 ... -| 3.224515243534819 -| 2.791117213058638 -| 0.000000000000000 error--> gawk: pi.awk:6: fatal: division by zero attempted

Here is an additional example where the inaccuracies in internal representations yield an unexpected result:

$gawk 'BEGIN {>for (d = 1.1; d <= 1.5; d += 0.1) # loop five times (?)>i++>print i>}'-| 4

Can computation using arbitrary precision help with the previous examples? If you are impatient to know, see Exact Arithmetic.

Instead of arbitrary precision floating-point arithmetic,
often all you need is an adjustment of your logic
or a different order for the operations in your calculation.
The stability and the accuracy of the computation of the constant *pi*
in the earlier example can be enhanced by using the following
simple algebraic transformation:

(sqrt(x * x + 1) - 1) / x = x / (sqrt(x * x + 1) + 1)

After making this, change the program does converge to
*pi* in under 30 iterations:

$gawk -f pi2.awk-| 3.215390309173473 -| 3.159659942097501 -| 3.146086215131436 -| 3.142714599645370 -| 3.141873049979825 ... -| 3.141592653589797 -| 3.141592653589797

There is no need to be unduly suspicious about the results from floating-point arithmetic. The lesson to remember is that floating-point arithmetic is always more complex than arithmetic using pencil and paper. In order to take advantage of the power of computer floating-point, you need to know its limitations and work within them. For most casual use of floating-point arithmetic, you will often get the expected result in the end if you simply round the display of your final results to the correct number of significant decimal digits.

As general advice, avoid presenting numerical data in a manner that implies better precision than is actually the case.

[1] One recommended title is Numerical Computing with IEEE Floating Point Arithmetic, Michael L. Overton, Society for Industrial and Applied Mathematics, 2004. ISBN: 0-89871-482-6, ISBN-13: 978-0-89871-482-1. See http://www.cs.nyu.edu/cs/faculty/overton/book.

[2] If you
are interested in other tools that perform arbitrary precision arithmetic,
you may want to investigate the POSIX `bc` tool. See
the POSIX specification for it, for more information.