Next: , Previous: Floating-point Constants, Up: Arbitrary Precision Floats


15.4.4 Changing the Precision of a Number

The point is that in any variable-precision package, a decision is made on how to treat numbers given as data, or arising in intermediate results, which are represented in floating-point format to a precision lower than working precision. Do we promote them to full membership of the high-precision club, or do we treat them and all their associates as second-class citizens? Sometimes the first course is proper, sometimes the second, and it takes careful analysis to tell which.

Dirk Laurie1

gawk does not implicitly modify the precision of any previously computed results when the working precision is changed with an assignment to PREC. The precision of a number is always the one that was used at the time of its creation, and there is no way for the user to explicitly change it afterwards. However, since the result of a floating-point arithmetic operation is always an arbitrary precision floating-point value—with a precision set by the value of PREC—one of the following workarounds effectively accomplishes the desired behavior:

     x = x + 0.0

or:

     x += 0.0

Footnotes

[1] Dirk Laurie. Variable-precision Arithmetic Considered Perilous — A Detective Story. Electronic Transactions on Numerical Analysis. Volume 28, pp. 168-173, 2008.