FIX: Math Problem with INT() Function

Last reviewed: October 20, 1997
Article ID: Q131578

3.00 3.00b WINDOWS kbprg kbbuglist kbfixlist

The information in this article applies to:

Microsoft Visual FoxPro for Windows, versions 3.0, 3.0b

SYMPTOMS

The INT(nExpression) Function evaluates a numeric expression and returns incorrect values.

RESOLUTION

To work around this problem, change the SET DECIMAL setting to a larger more appropriate value such as 10 (the default is 2).

STATUS

Microsoft has confirmed this to be a problem in the Microsoft products listed at the beginning of this article. This problem has been fixed in Visual FoxPro 5.0 for Windows.

MORE INFORMATION

Steps to Reproduce Problem

Run the following code in Visual FoxPro:

   pnum1 = 1000000
   pnum2 = 999999

   SET DECI TO 2       && this is default

   ? int(pnum2/pnum1)  && result is 1 it should be 0

   SET DECI TO 10

   ? int(pnum2/pnum1)  && result is 0, which is correct

KBCategory: kbprg kbbuglist kbfixlist
KBSubcategory: FxprgGeneral fixlist5.00 VFoxWin buglist3.00 buglist3.00b
KBQ
Additional reference words: 3.00 3.00b
Keywords : buglist3.00 buglist3.00b fixlist5.00 FxprgGeneral vfoxwin kbbuglist kbfixlist kbprg
Version : 3.00 3.00b
Platform : WINDOWS
Solution Type : kbfix

THE INFORMATION PROVIDED IN THE MICROSOFT KNOWLEDGE BASE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. MICROSOFT DISCLAIMS ALL WARRANTIES, EITHER EXPRESS OR IMPLIED, INCLUDING THE WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL MICROSOFT CORPORATION OR ITS SUPPLIERS BE LIABLE FOR ANY DAMAGES WHATSOEVER INCLUDING DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, LOSS OF BUSINESS PROFITS OR SPECIAL DAMAGES, EVEN IF MICROSOFT CORPORATION OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME STATES DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY FOR CONSEQUENTIAL OR INCIDENTAL DAMAGES SO THE FOREGOING LIMITATION MAY NOT APPLY.

Last reviewed: October 20, 1997
© 1998 Microsoft Corporation. All rights reserved. Terms of Use.