ACC: Rounding Errors Using Floating-Point NumbersLast reviewed: August 29, 1997Article ID: Q111781 |
The information in this article applies to:
SYMPTOMSNovice: Requires knowledge of the user interface on single- user computers. When you calculate using floating-point numbers, the result is not always what you expect. For example, if you use the CInt() (convert to integer) function in the calculation
CInt(3.555 * 100)you receive 356 as the result, as you expect. However, the calculation
CInt(4.555 * 100)results in 455 instead of 456.
CAUSEErrors similar to the example above occur in any programming language that uses floating-point numbers. This is because decimal fractions do not always have exact binary equivalents, which can result in rounding errors.
RESOLUTIONThere are several methods you can use to avoid rounding errors when you are using floating-point numbers:
|
Additional query words: roundoff
© 1998 Microsoft Corporation. All rights reserved. Terms of Use. |