BUG: FoxPro Math-Precision Problems with SET DECIMALS

• Microsoft Visual FoxPro for Windows, versions 3.0, 3.0b, 5.0, 5.0a
• Microsoft FoxPro for Windows, versions 2.5x, 2.6x
• Microsoft FoxPro for MS-DOS, version 2.0, 2.5x, 2.6x

SYMPTOMS

SET DECIMALS TO 2 changes the result when a numeric expression that evaluates to a decimal value is used in a comparison. For example, type the following in the Command window:

   SET DECIMAL TO 18
   ? 12/8888888 = 0     && FoxPro returns FALSE

   SET DECIMALS TO 2
   ? 12/8888888 = 0     && FoxPro returns TRUE

In FoxPro version 2.6a, a similar decimal-precision problem occurs when any whole number is used in math problems. For example:

   SET DECIMALS TO 18
   ?12/8888888       && dBASE returns 0.00000135... but FoxPro returns 0

   ?12/88888888      && dBASE returns 0.0000000000... but FoxPro returns 0

   ?12.0/8888888     && dBASE returns 0.00000135... but FoxPro returns 0

   SET DECIMALS TO 18
   ?12/8888888       && FoxPro returns 0.000001350000135000

   ?12/88888888      && FoxPro returns 0

   ?12.0/8888888     && FoxPro returns 0.000001350000135000

RESOLUTION

If you need greater precision in FoxPro, set the SET DECIMALS command to a higher value. For greater precision with constants, replace the constants with memory variables, so that FoxPro performs the calculation at run time.

If you must use constants, express them in the precision that is desired. For example, use this expression

   ?12.0000000/88888888  && FoxPro returns 0.00000135...

   ?12/88888888  && FoxPro returns 0.0000000000...

STATUS

Microsoft has confirmed this to be a problem in the Microsoft products listed at the beginning of this article. We are researching this problem and will post new information here in the Microsoft Knowledge Base as it becomes available.

MORE INFORMATION

NOTE: FoxPro will return a zero when the first significant digit is in the seventh position to the right of the decimal; for example, 0.0000001.