FLP00-A. Understand the limitations of floating point numbers

The C programming languages provides the ability to use floating point numbers for calculations. C99 specifies an overall library that a conforming implementation needs to follow for floating point numbers, but makes few guarantees about the specific underlying floating point representation. Because of the preexistence of competing floating point systems, C99 uses intentionally vague language when dealing with the requirements of floating point values.

By definition, a floating point number is of finite precision, and regardless of the underlying implementation, is prone to errors associated with rounding (see FLP01-A. Take care in rearranging floating point expressions and FLP04-A. Consider avoiding floating point numbers when precise computation is needed).

Though a fully conforming implementation is free to create its own floating point system, there are two systems which garner overwhelming popularity. The first, used by the default options of Microsoft Visual Studio and GCC on Intel architectures, is IEEE 754. The second is the system used natively by derivative architectures of IBM System/360, commonly known as the "IBM floating point representation," or "IBM/370." Each of these systems have differing precisions and ranges of representable values. Thus, they do not represent all of the same values, are not binary compatible, and have differing associated error rates.

Because of a lack of guarantees about precision and range of the underlying floating point system, care must be taken that if the code is intended to be portable no assumptions are made about either and that even if the code is not intended to be portable, the chosen compiler's behavior must be well understood at all compiler optimization levels.

Here is a simple illustration of precision limitations. The following code prints the decimal representation of 1/3, to 50 decimal places. Ideally, it would print 50 3's.

#include <stdio.h>

int main() {
  float f = 1.0 / 3.0;
  printf("Float is %.50f\n", f);
  return 0;
}

On 64-bit Linux, with gcc 4.1, this produces:

Float is 0.33333334326744079589843750000000000000000000000000

On Windows XP, with MSVC++ 9.0, this produces:

Float is 0.33333334326744080000000000000000000000000000000000

Risk Analysis

Failing to understand the limitations of floating point numbers can result in unexpected mathematical results and exceptional conditions, possibly resulting in a violation of data integrity.

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

References

[[IEEE 754 2006]]
[[ISO/IEC 9899-1999]] Section 5.2.4.2.2, "Characteristics of floating types <float.h>"

05. Floating Point (FLP)      05. Floating Point (FLP)       FLP01-A. Take care in rearranging floating point expressions