Math Coprocessor May Use Different Level of Precision

ID: Q57298


The information in this article applies to:
  • Microsoft Excel for Windows, versions 2.x, 3.x, 4.x, 5.0, 5.0c
  • Microsoft Excel for Windows 95, version 7.0


SUMMARY

In Microsoft Excel, when you perform calculations using numbers that contain decimals, you may receive slightly different (more accurate) results with a math coprocessor present.


MORE INFORMATION

If a math coprocessor is not present, Microsoft Excel uses math coprocessor emulation for its calculations. Calculations using a math coprocessor may use higher precision and therefore may return a slightly different number than the same calculation using the emulation.

The difference between emulation and coprocessor is that emulation is done entirely with double precision floats (8 bytes). The coprocessor may convert to a higher precision format (extended precision floats (10 bytes)) internally, but we always interface with it with double precision floats.

Therefore although a math coprocessor may give slightly higher precision, it doesn't always do so. Emulation always gives at least double precision accuracy.

For additional information, please see the following article(s) in the Microsoft Knowledge Base:

Q78113 : Excel: Floating-Point Arithmetic May Give Inaccurate Results

Additional query words: 2.00 2.01 2.10 2.20 3.00 4.00 4.00a 5.00c FPU

Keywords :
Version :
Platform :
Issue type :


Last Reviewed: March 15, 1999
© 2000 Microsoft Corporation. All rights reserved. Terms of Use.