WD98: Table Sum Rounded Incorrectly on Large NumbersLast reviewed: February 17, 1998Article ID: Q177893 |
The information in this article applies to:
SYMPTOMSWhen you sum a number in a table that is greater than 15 decimal places, the number is rounded to the nearest 10. For example, the following 16 digit number rounds to the nearest 10th.
6553565535655351 + 1 = 6553565535655350 CAUSEThis is a limitation of the floating-point processor. The exact behavior of a floating-point calculation is hardware dependent. Table, Formula, OK |
Additional query words: OFF98 floating point
© 1998 Microsoft Corporation. All rights reserved. Terms of Use. |