BUG: CALCULATE SUM() Results Differ Based on SET DECIMALLast reviewed: June 27, 1995Article ID: Q113209 |
The information in this article applies to:
SYMPTOMSIssuing the CALCULATE SUM() command with a field as the argument to SUM() will return values other than 0 (zero) past the second decimal place if SET DECIMALS TO is set to greater than 2. However, this behavior should never occur if the field is defined as having only two decimals.
STATUSMicrosoft has confirmed this to be a problem in the Microsoft products listed at the beginning of this article. We are researching this problem and will post new information here in the Microsoft Knowledge Base as it becomes available.
MORE INFORMATION
Steps to Reproduce ProblemRun the following code from a program:
SET SAFETY OFF
SET TALK OFF
=RAND(-1)
CREATE TABLE test (VALUE N(10,2))
FOR i = 1 TO 2000
INSERT INTO test (VALUE) VALUES (RAND()*100)
NEXT i
SET DECIMAL TO 2
CALCULATE SUM(VALUE) TO val1
SET DECIMAL TO 14
CALCULATE SUM(VALUE) TO val2
?"VAL1 ="
??val1
?"VAL2 ="
??val2
SET TALK ON
SET SAFETY ON
The output that this program produces will look something like the
following, although the values will be different:
VAL1 = 7483984.34 VAL2 = 7483984.33999998000000 -or- VAL1 = 7483984.34 VAL2 = 7483984.34000000001000NOTE: You may have to run the program several times before you will see this behavior. However, the output should look like the following:
VAL1 = 7483984.34 VAL2 = 7483984.34000000000000VAL2 should have zeros only from the third decimal place on because TEST.VALUE is defined in the table as being only two decimal places wide.
|
Additional reference words: FoxMac FoxDos FoxWin 2.00 2.50 2.50a 2.50b
© 1998 Microsoft Corporation. All rights reserved. Terms of Use. |