Numeric Limits/en

Integer Arithmetic

Expecco supports arbitrary precision integer arithmetic, arbitrary precision fractions and limited precision floating point numbers.

For integer operations, there is no overflow or error in the result for any legal operation. I.e. operations on two big numbers deliver a correct result (in contrast to many other programming languages; this is a feature of the underlying Smalltalk runtime environment):

4294967295 "(0xFFFFFFFF)" + 1
    -> 4294967296 "(0x100000000)"

18446744073709551615 "(0xFFFFFFFFFFFFFFFF)" + 1
    -> 18446744073709551616 "(0x10000000000000000)"

when dividing integers, the "/" operator will deliver an exact result, possibly as a fraction:

5 / 3 -> 5/3

and the truncating division "//" will deliver an integer, truncated towards negative infinity (i.e. the next smaller integer):

5 // 3 -> 1
-5 // 3 -> -3

The modulo operator "\\" provides the remainder, such that:

(a // b) + (a \\ b) = a

There is also a truncating division operator which truncates towards zero, and a corresponding remainder operator, for which:

(a quo: b) + (a rem: b) = a

For positive a and b, the two operator pairs deliver the same result.

In addition, the usual ceiling, floor and rounding operations are available (both on fractions and on limited precision reals):

(5 / 3) ceiling -> 2
(5 / 3) floor -> 1
(5 / 3) rounded -> 2
(5 / 3) roundTo: 0.1  -> 1.7
(5 / 3) roundTo: 0.01  -> 1.67

(-5 / 3) ceiling -> -1
(-5 / 3) floor -> -2
(-5 / 3) rounded -> -2
(-5 / 3) roundTo: 0.1 -> -1.7

Inexact Float and Double Numbers

Floating point numbers are inherently inexact. This is not a problem specific to expecco, but inherent to the way floating point numbers are represented. See "What Every Computer Scientist Should Know About Floating-Point Arithmetic"
(many numbers cannot be represented exactly as a sum of powers of 2, and the floating point number has an error in the last significant bit).

Such errors will even accumulate, with every math operation performed on it. For example, the decimal 0.1 cannot be exactly represented as a floating point number, and is actually 0.099999... with an error in the last bit. Adding this multiple times will result in a larger error in the final result:

1.0 - (0.1 + 0.1 + 0.1 ...(10 times)... + 0.1) -> 1.11022302462516E-16

The print functions will usually try to compensate for an error in the last bit, showing "0.1" although in reality, it is "0.9999...".

No such rounding error are encountered, if fractions are used:

1 - ((1/10) + (1/10) + (1/10) ...(10 times)... + (1/10)) -> 0

Be aware of such errors, and do not compare floating point numbers for equality/inequality. Instead either use range-compares and/or use the special "compare-almost-equal" functions, where the number of bits of acceptable error can be specified. Expecco provides such functions both for elementary code and in the standard block library.

Also, for this reason, do not compute money values using floats or doubles. Instead, use instances of FixedPoint, which are fractions with a mutiple-of-10 denominator, and which compute exact results. You will loose a cent/penny here and there, if you use floats/doubles on big budgets.

Trigonometric and other Math Functions

Trigonometric and other math functions (sqrt, log, exp) will first convert the number to a limited precision real number (a C-double), and therefore may have a limited input value range and also generate inexact results.

For example, you will not get a valid result for:

10000 factorial sin

because it is not possible to represent that large number as real number. (expecco will signal a domain error, as the input to sin will be +INF)

Also, the result from:

(9 / 64) sqrt

will be the (inexact) 0.75 (a double), instead of the exact 3/4 (a fraction). (the error is (0.75 - x), which is roughly 1.11022302462516E-16)

Different Results on Different CPUs

Since floating point arithmetic is performed by the underlying CPU hardware, different results (in the least significant bit) can be returned from math operations on different CPUs or even different versions (steppings) of the same CPU architecture. This applies especially to trigonometric and other math functions. Be prepared for this, and use the "almost-equal" comparison functions when results are to be verified.

Higher Precision Numbers

Expecco supports various inexact real formats, with different precision (i.e. number of mantissa bits):

Name                         overall   exponent mantissa  decimal   Smalltalk
                              size       size     size   precision  class name

IEEE single precision floats  32 bit     8        24       6        ShortFloat 
IEEE double precision floats  64 bit     11       53       15       Float
IEEE extended prec. floats.   80/128 bit 15       64/112   19/34    LongFloat   (1)
quad double                   4*64 bit   11       200      60       QDouble     (2)
arbitrary                     any        any      any      any      LargeFloat  (2)

(1) on x86/x64 machines, LongFloats are represented as 80bit extended floats with 64bit mantissa; on other CPUs, these might be represented as 128bit quadFloats with 112 bit mantissa.
(2) these are currently provided as a preview feature without warranty.

Notice that the use of any but double precision floats (which are directly supported by the machine) comes at a price. The speed of operations degrades from double -> single -> extended -> quad double -> largeFloat.

---

Copyright © 2014-2024 eXept Software AG