BUG: CALCULATE SUM() Results Differ Based on SET DECIMALLast reviewed: June 27, 1995Article ID: Q113209 |
The information in this article applies to:
SYMPTOMSIssuing the CALCULATE SUM() command with a field as the argument to SUM() will return values other than 0 (zero) past the second decimal place if SET DECIMALS TO is set to greater than 2. However, this behavior should never occur if the field is defined as having only two decimals.
STATUSMicrosoft has confirmed this to be a problem in the Microsoft products listed at the beginning of this article. We are researching this problem and will post new information here in the Microsoft Knowledge Base as it becomes available.
MORE INFORMATION
Steps to Reproduce ProblemRun the following code from a program:
SET SAFETY OFF SET TALK OFF =RAND(-1) CREATE TABLE test (VALUE N(10,2)) FOR i = 1 TO 2000 INSERT INTO test (VALUE) VALUES (RAND()*100) NEXT i SET DECIMAL TO 2 CALCULATE SUM(VALUE) TO val1 SET DECIMAL TO 14 CALCULATE SUM(VALUE) TO val2 ?"VAL1 =" ??val1 ?"VAL2 =" ??val2 SET TALK ON SET SAFETY ONThe output that this program produces will look something like the following, although the values will be different:
VAL1 = 7483984.34 VAL2 = 7483984.33999998000000 -or- VAL1 = 7483984.34 VAL2 = 7483984.34000000001000NOTE: You may have to run the program several times before you will see this behavior. However, the output should look like the following:
VAL1 = 7483984.34 VAL2 = 7483984.34000000000000VAL2 should have zeros only from the third decimal place on because TEST.VALUE is defined in the table as being only two decimal places wide.
|
Additional reference words: FoxMac FoxDos FoxWin 2.00 2.50 2.50a 2.50b
© 1998 Microsoft Corporation. All rights reserved. Terms of Use. |