Understand integer conversion rules

The runtime behavior of integers in C is not always well-understood by programmers, leading to exploitable vulnerabilities.

  For example, assume the following code is compiled and executed on IA-32:

signed char sc = SCHAR_MAX;
unsigned char uc = UCHAR_MAX;
signed long long sll = sc + uc;

Both the signed char sc and the unsigned char uc are subject to integer promotions in this example. Because all values of the original types can be represented as int, both values are automatically converted to int as part of the integer promotions. Further conversions are possible, if the types of these variables are not equivalent as a result of the "usual arithmetic conversions".  The actual addition operation in this case takes place between the two 32-bit int values. This operation is not influenced by the resulting value is stored in a signed long long integer. The 32-bit value resulting from the addition is simply sign-extended to 64-bits after the addition operation has concluded.

Assuming that the precision of signed char is 7 bits, and the precision of unsigned char is 8 bits, this operation is perfectly safe. However, if the compiler represents the signed char and unsigned char types using 31 and 32 bit precision (respectively), the variable uc would need be converted to unsigned int instead of signed int. As a result of the usual arithmetic conversions, the signed int is converted to unsigned and the addition takes place between the two unsigned int values. Also, because uc is equal to UCHAR_MAX which is equal to UINT_MAX in this example, the addition will result in an overflow. The resulting value is then zero-extended to fit into the 64-bit storage allocated by sll.

Non-compliant Code Example 1

In the following non-compliant code example, cBlocks is multiplied by 16 and the result is stored in the unsigned long long int alloc.  The result of this multiplication can overflow because it is a 32 bit operation and the resulting value stored in alloc invalid.

void* AllocBlocks(size_t cBlocks) {
  if (cBlocks == 0) return NULL;
  unsigned long long alloc = cBlocks * 16;
  return (alloc < UINT_MAX)
    ? malloc(cBlocks * 16)
    : NULL;
}

Compliant Solution 1

On architectures where unsigned long long int is guaranteed to have 2x the number of bits as size_tupcast the variable used in the multiplication to a 64-bit value.  This ensures the multiplication operation is performed

void* AllocBlocks(size_t cBlocks) {
  if (cBlocks == 0) return NULL;
  unsigned long long alloc =
           (unsigned long long)cBlocks*16;
  return (alloc < UINT_MAX)
    ? malloc(cBlocks * 16)
    : NULL;
}

The assumption concerning the relationship of unsigned long long int and size_t must be document in the header for each file that depends upon this assumption for correct execution.

Exceptions

Unsigned integers can be allowed to exhibit modulo behavior if and only if

1. the variable declaration is clearly commented as supporting modulo behavior
2. each operation on that integer is also clearly commented as supporting modulo behavior
3. if the integer exhibiting modulo behavior contributes to the value of an integer not marked as exhibiting modulo behavior, the resulting integer must obey this rule.

Consequences

Improper range checking can lead to buffer overflows and the execution of arbitary code by an attacker.

References

• Seacord 05 Chapter 5 Integers