Numeric precision in Microsoft Excel

As with other spreadsheets, <a href="/facts/Microsoft_Excel/Ezal6G75">Microsoft Excel</a> works only to limited accuracy because it retains only a certain number of figures to describe numbers (it has limited <a href="/facts/Arithmetic_precision/RUfYtpgo">precision</a>). With some exceptions regarding erroneous values, infinities, and denormalized numbers, Excel calculates in <a href="/facts/Double-precision_floating-point_format/JYyXXYFM">double-precision floating-point format</a> from the <a href="/facts/IEEE_754-2008/gqqJPoYC">IEEE 754 specification</a> (besides numbers, Excel uses a few other data types). Although Excel allows display of up to 30 decimal places, its precision for any specific number is no more than 15 <a href="/facts/Significant_figures/RUfYtpgo">significant figures</a>, and calculations may have an accuracy that is even less due to five issues: <a href="/facts/Round-off_error/aF9N8X4I">round off</a>,
<a href="/facts/Truncation/8mEHh0qp">truncation</a>, and <a href="/facts/Binary_numeral_system/a6JDSRPs">binary storage</a>, accumulation of the deviations of the operands in calculations, and worst: cancellation at subtractions resp. 'Catastrophic cancellation' at subtraction of values with similar magnitude.

Numeric precision in Microsoft Excel open-in-new

Numeric precision in Microsoft Excel