[Jprogramming] dyad ": rounding error

[John Randall]

Joey K Tuttle wrote:
> Ah, the perils of floating point numbers....
>
> Even though you say you would never use rational
> numbers in calculations, most accountants do.
> For example, the calculators you mentioned in
> your points often do fixed calculations with
> the number of decimals set as a fixed option.
>
I guess one should note that all floating-point numbers represent
rational numbers, but they do not always obey the laws of rational
arithmetic.  Fixed-point numbers are no different.

> Similarly, many accounting programs store amounts
> of money in cents. e.g. in phone call records, the
> rates (cents/minute) are usually stored as 10000
> times the dollar amount - and so on. Formatting
> the final numbers is only a part of the problem...
>

Excel attempts to make binary floating-point arithmetic look like
decimal by using cosmetic rounding: see

http://www.cs.berkeley.edu/~wkahan/Mindless.pdf#page=3

Best wishes,

John