Numeric Limits/en
Inhaltsverzeichnis
Integer Arithmetic[Bearbeiten]
Expecco supports arbitrary precision integer arithmetic, arbitrary precision fractions and limited precision floating point numbers.
For integer operations, there is no overflow or error in the result for any legal operation. I.e. operations on two big numbers deliver a correct result:
4294967295 (0xFFFFFFFF) + 1 -> 4294967296 (0x100000000)
18446744073709551615 (0xFFFFFFFFFFFFFFFF) + 1 -> 18446744073709551616 (0x10000000000000000)
when dividing integers, the "/" operator will deliver an exact result, possibly as a fraction:
5 / 3 -> 5/3
and the truncating division "//" will deliver an integer, truncated towards negative infinity (i.e. the next smaller integer):
5 // 3 -> 2 -5 // 3 -> -3
Inexact Float and Double Numbers[Bearbeiten]
Floating point numbers are inherently inexact. This is not a problem specific to expecco, but inherent to the way, floating point numbers are represented (many numbers cannot be represented exactly as a sum of powers of 2, and the floating point number has an error in the last significant bit).
Such errors will even accumulate, with every math operation performed on it. For example, the decimal 0.1 cannot be exactly represented as a floating point number, and is actually 0.099999... with an error in the last but. Adding this multiple times will result in a larger error in the final result:
1.0 - (0.1 + 0.1 + 0.1 ...(10 times)... + 0.1) -> 1.11022302462516E-16
The print functions will usually try to compensate for an error in the last bit, showing "0.1" although in reality, it is "0.9999...".
No such rounding error are encountered, if fractions are used:
1 - ((1/10) + (1/10) + (1/10) ...(10 times)... + (1/10)) -> 0
Be aware of such errors, and do not compare floating point numbers for equality/inequality. Instead either use range-compares and/or use the special "compare-almost-equal" functions, where the number of bits of acceptable error can be specified. Expecco provides such functions both for elementary code and in the standard block library.
Also, for this reason, do not compute money values using floats or doubles. Instead, use instances of FixedPoint, which are fractions with a mutiple-of-10 denominator, and which compute exact results. You will loose a cent/penny here and there, if you use floats/doubles on big budgets.
Trigonometric and other Math Functions[Bearbeiten]
Trigonometric and other math functions (sqrt, log, exp) will first convert the number to a limited precision real number (a C-double), and therefore may have a limited input value range and also generate inexact results.
For example, you will not get a valid result for:
10000 factorial sin
because it is not possible to represent that large number as real number. (expecco will signal a domain error, as the input to sin will be +INF)
Also, the result from:
(9 / 64) sqrt
will be the (inexact) 0.75 (a double), instead of the exact 3/4 (a fraction). (the error is (0.75 - x), which is roughly 1.11022302462516E-16)
Different Results on Different CPUs[Bearbeiten]
Since floating point arithmetic is performed by the underlying CPU hardware, different results (in the least significant bit) can be returned from math operations on different CPUs or even different versions (steppings) of the same CPU architecture. This applies especially to trigonometric and other math functions. Be prepared for this, and use the "almost-equal" comparison functions when results are to be verified.
