PRB: Overflow Doesn't Occur Using Pentium Pro or Pentium IILast reviewed: September 30, 1997Article ID: Q168610 |
The information in this article applies to:
SYMPTOMSWhen converting a floating point value to an integer value on Intel Pentium Pro and Pentium II processors, a negative value much less than the smallest negative integer does not cause a "Run-Time Error '6': Overflow," as would be expected. This problem is actually caused by an erratum in the Pentium Pro and Pentium II processors. The following code illustrates this problem:
Debug.Print CInt(-2.59615E+33) 'Where -2.59615E+33 = -2,596,150,000,000,000,000,000,000,000,000,000Normally, this code will generate "Run-Time Error '6': Overflow." However, on Pentium Pro and Pentium II processors, this code does not generate an overflow. For conversion to type Integer, the failure to report an overflow occurs only when the value is in the range -1.84 x 10^19 to -2.60 x 10^33; that is, between:
-18,400,000,000,000,000,000 -and- -2,600,000,000,000,000,000,000,000,000,000,000Within the range above, fewer than one out of every 65,000 values is affected. For conversion to type Long, the failure to report an overflow occurs only when the value is in the range -1.84 x 10^19 to -3.96 x 10^28; that is, between:
-18,400,000,000,000,000,000 -and- -39,600,000,000,000,000,000,000,000,000.Within the range above, fewer than one out of every 4,000,000,000 values is affected. For this problem to be relevant to the Visual Basic developer, the application must meet the following conditions:
CAUSEIntel Corporation has identified an erratum in the Pentium Pro and Pentium II processors relating to the conversion of floating point values to signed integer values when using the Floating Point Integer Store (FIST) instruction available with these processors. The FIST instruction converts floating point numbers to 16-, 32-, or 64-bit signed integers. Because the range of a floating point number is larger than any of these formats, some floating point numbers cannot be converted to integers. When attempting to convert a floating point number that is too large to an integer, the processor should signal an "Invalid Operation" exception. Visual Basic uses this signal to report "Run-Time Error '6': Overflow." The erratum in the Pentium Pro and Pentium II processors causes them to not signal the "Invalid Operation" exception for certain numbers in the ranges given above.
RESOLUTIONAn application may be affected by this problem if all three of the conditions listed under the SYMPTOMS section above are met. In that case, the application must be modified to explicitly test for large negative values and raise the exception itself. This must be done at any point that a floating point number, Single or Double, is converted to Integer or Long, including the following situations:
If FloatingPointValue < -1E+18 Then Error 6For example:
Dim IntegerValue As Integer Dim FloatingPointValue As Single FloatingPointValue = -2.59615E+33 If FloatingPointValue < -1E+18 Then Error 6 IntegerValue = FloatingPointValueThis test and conversion may be encapsulated into a pair of functions, as follows:
Public Function FloatToInt(Expression as Variant) As Integer If Expression < -1E+18 Then Error 6 FloatToInt = Expression End Function Public Function FloatToLong(Expression as Variant) As Long If Expression < -1E+18 Then Error 6 FloatToLong = Expression End FunctionUsing the FloatToInt function above, the previous example becomes:
Dim IntegerValue As Integer Dim FloatingPointValue As Single FloatingPointValue = -2.59615E+33 IntegerValue = FloatToInt(FloatingPointValue) STATUSMicrosoft has confirmed that this erratum in the processor affects the Microsoft products listed at the beginning of this article. We are researching this problem and will post new information here in the Microsoft Knowledge Base as it becomes available.
MORE INFORMATIONFor more details on this erratum in the Pentium Pro processor, refer to:
http://developer.intel.com/design/pro/update/pprosu.htmFor more details on this erratum in the Pentium II processor, refer to:
http://developer.intel.com/design/PentiumII/update/ppiisu.htmIntel’s documentation on this erratum will note additional ranges of values in which the "Invalid Operation" exception is not set. Visual Basic is not affected by these additional value ranges and it will report the overflow error correctly for them.
REFERENCESFor additional information on a similar problem that affected earlier versions of Visual Basic, see the following article in the Microsoft Knowledge Base:
ARTICLE-ID: Q126455 TITLE : How to Avoid Rounding & Overflow Probs on Pentium Processors Keywords : RefsThird VB4ALL VB4WIN vb5all VBKB3rdParty vbwin GnrlVb kb3rdparty kbhw kbprg Technology : kbvba Version : WINDOWS:4.0 5.0 7.0 7.0a 7.0b 97 Platform : NT Win95 WINDOWS Issue type : kbprb |
================================================================================
© 1998 Microsoft Corporation. All rights reserved. Terms of Use. |