Next: Modes Tutorial Exercise 2, Previous: Algebraic Entry Tutorial Exercise 3, Up: Answers to Exercises [Contents][Index]

Calc always stores its floating-point numbers in decimal,
so even though one-third has
an exact base-3 representation (‘`3#0.1`’), it is still stored as
0.3333333 (chopped off after 12 or however many decimal digits) inside
the calculator’s memory. When this inexact number is converted back
to base 3 for display, it may still be slightly inexact. When we
multiply this number by 3, we get 0.999999, also an inexact value.

When Calc displays a number in base 3, it has to decide how many digits
to show. If the current precision is 12 (decimal) digits, that corresponds
to ‘`12 / log10(3) = 25.15`’ base-3 digits. Because 25.15 is not an
exact integer, Calc shows only 25 digits, with the result that stored
numbers carry a little bit of extra information that may not show up on
the screen. When Joe entered ‘`3#0.2`’, the stored number 0.666666
happened to round to a pleasing value when it lost that last 0.15 of a
digit, but it was still inexact in Calc’s memory. When he divided by 2,
he still got the dreaded inexact value 0.333333. (Actually, he divided
0.666667 by 2 to get 0.333334, which is why he got something a little
higher than `3#0.1`

instead of a little lower.)

If Joe didn’t want to be bothered with all this, he could have typed
`M-24 d n` to display with one less digit than the default. (If
you give `d n` a negative argument, it uses default-minus-that,
so `M-- d n` would be an easier way to get the same effect.) Those
inexact results would still be lurking there, but they would now be
rounded to nice, natural-looking values for display purposes. (Remember,
‘`0.022222`’ in base 3 is like ‘`0.099999`’ in base 10; rounding
off one digit will round the number up to ‘`0.1`’.) Depending on the
nature of your work, this hiding of the inexactness may be a benefit or
a danger. With the `d n` command, Calc gives you the choice.

Incidentally, another consequence of all this is that if you type
`M-30 d n` to display more digits than are “really there,”
you’ll see garbage digits at the end of the number. (In decimal
display mode, with decimally-stored numbers, these garbage digits are
always zero so they vanish and you don’t notice them.) Because Calc
rounds off that 0.15 digit, there is the danger that two numbers could
be slightly different internally but still look the same. If you feel
uneasy about this, set the `d n` precision to be a little higher
than normal; you’ll get ugly garbage digits, but you’ll always be able
to tell two distinct numbers apart.

An interesting side note is that most computers store their
floating-point numbers in binary, and convert to decimal for display.
Thus everyday programs have the same problem: Decimal 0.1 cannot be
represented exactly in binary (try it: `0.1 d 2`), so ‘`0.1 * 10`’
comes out as an inexact approximation to 1 on some machines (though
they generally arrange to hide it from you by rounding off one digit as
we did above). Because Calc works in decimal instead of binary, you can
be sure that numbers that look exact *are* exact as long as you stay
in decimal display mode.

It’s not hard to show that any number that can be represented exactly in binary, octal, or hexadecimal is also exact in decimal, so the kinds of problems we saw in this exercise are likely to be severe only when you use a relatively unusual radix like 3.