The runtime behavior of integers in C is not always well-understood by programmers, leading to exploitable vulnerabilities. For example, assume the following code is compiled and executed on IA-32:
signed char sc = SCHAR_MAX; unsigned char uc = UCHAR_MAX; signed long long sll = sc + uc;
Both the signed char sc and the unsigned char uc are subject to integer promotions in this example. Because all values of the original types can be represented as int, both values are automatically converted to int as part of the integer promotions. Further conversions are possible, if the types of these variables are not equivalent as a result of the "usual arithmetic conversions". The actual addition operation in this case takes place between the two 32-bit int values. This operation is not influenced at all by the fact that the resulting value is stored in a signed long long integer. The 32-bit value resulting from the addition is simply sign-extended to 64-bits after the addition operation has concluded.
Assuming that the precision of signed char is 7 bits, and the precision of unsigned char is 8 bits, this operation is perfectly safe. However, if the compiler represents the signed and unsigned char types using 31 and 32 bit precision (respectively), the variable uc would need be converted to unsigned int instead of signed int. As a result of the usual arithmetic conversions, the signed int would then be converted to unsigned and the addition would take place between the two unsigned int values. Also, because uc is equal to UCHAR_MAX which is equal to UINT_MAX in this example, the addition will result in an overflow. The resulting value is then zero-extended to fit into the 64-bit storage allocated by sll.
Addition
Addition in C is between two operands of arithmetic type, or between a pointer to an object type and an integer type. (Incrementing is equivalent to adding one.)
Non-compliant Code Example
The following code will result in an unsigned integer overflow during the addition of the unsigned operands ui1 and ui2. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner which could lead to an exploitable vulnerability.
unsigned int sum; unsigned int ui1 = UINT_MAX; unsigned int ui2 = 1; sum = ui1 + ui2;
Compliant Solution
The following compliant solution tests the suspect addition operation to guarantee there is no possibility of unsigned overflow. In this particular case, an overflow condition is present and the error_handler() method is invoked.
unsigned int sum;
unsigned int ui1 = UINT_MAX;
unsigned int ui2 = 1;
if (~ui1 < ui2){
error_handler("Overflow Error", NULL, EOVERFLOW);
}
sum = ui1 + ui2;
Subtraction
Subtraction in C is between two operands of arithmetic type, two pointers to qualified or unqualified versions of compatible object types, or between a pointer to an object type and an integer type. (Decrementing is equivalent to subtracting one.)
Non-compliant Code Example
The following code will result in a signed integer overflow during the subtraction of the signed operands si1 and si2. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner which could lead to an exploitable vulnerability.
signed int result; signed int si1 = INT_MAX; signed int si2 = -1; result = si1 - si2;
Compliant Solution
The following compliant solution tests the suspect subtraction operation to guarantee there is no possibility of signed overflow. In this particular case, an overflow condition is present and the error_handler() method is invoked.
signed int result;
signed int si1 = INT_MAX;
signed int si2 = -1;
if ( ((si1^si2)&((si1-si2)^si1)) >> (sizeof(int)*CHAR_BIT-1) ){
error_handler("OVERFLOW ERROR", NULL, EOVERFLOW);
}
result = si1 - si2;
Multiplication
Multiplication in C is between two operands of arithmetic type.
Non-compliant Code Example
The following code will result in a signed integer overflow during the multiplication of the signed operands si1 and si2. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner which could lead to an exploitable vulnerability.
signed int result; signed int si1 = INT_MIN; signed int si2 = 2; result = si1 * si2;
Compliant Solution
The following compliant solution tests the suspect multiplication operation to guarantee there is no possibility of signed overflow. In this particular case, an overflow condition is present and the error_handler() method is invoked.
signed int result;
signed int si1 = INT_MIN;
signed int si2 = 2;
signed long long tmp = (signed long long)lhs * (signed long long)rhs;
if ( (tmp > INT_MAX) || (tmp < INT_MIN)) {
/* The product cannot fit in a 32-bit int */
error_handler("OVERFLOW ERROR", NULL, EOVERFLOW);
}
result = (int)tmp;
It is important to note that the above code is only compliant on systems where long long is atleast twice the size of int. On systems where this does not hold the following compliant solution may be used to ensure signed overflow does not occur.
signed int result;
signed int si1 = INT_MIN;
signed int si2 = 2;
if( si1> 0){
if( si2> 0){
if(si1> (INT_MAX / si2))
error_handler("OVERFLOW ERROR", NULL, EOVERFLOW);
} else {
if ( si2< (INT_MIN / si1))
error_handler("OVERFLOW ERROR", NULL, EOVERFLOW);
}
} else {
if(si2 > 0){
if( si1 < (INT_MIN / si2))
error_handler("OVERFLOW ERROR", NULL, EOVERFLOW);
} else {
if( (si1 != 0) && (si2 < (INT_MAX / si1)))
error_handler("OVERFLOW ERROR", NULL, EOVERFLOW);
}
}
result = si1 * si2;
Division
Division in C is between two operands of arithmetic type.
Non-compliant Code Example
The following code will result in a signed integer overflow during the division of the signed operands sl1 and sl2. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner which could lead to an exploitable vulnerability.
signed long sl1 = LONG_MIN; signed long sl2 = -1; signed long result; result = sl1 / sl2;
Compliant Solution
The following compliant solution tests the suspect division operation to guarantee there is no possibility of signed overflow. In this particular case, an overflow condition is present and the error_handler() method is invoked.
signed long sl1 = LONG_MIN;
signed long sl2 = -1;
signed long result;
if( (sl2 == 0) || ( (sl1 == LONG_MIN) && (sl2 == -1) ) ){
error_handler("ERROR OVERFLOW", NULL, EOVERFLOW);
return 0;
}
result = sl1 / sl2;
Unary Negation
The unary negation operator in C takes an operand of arithmetic type.
Non-compliant Code Example
The following code will result in a signed integer overflow during the unary negation of the signed operand si1. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner which could lead to an exploitable vulnerability.
signed int result; signed int si1 = INT_MIN; result = -si1;
Compliant Solution
The following compliant solution tests the suspect negation operation to guarantee there is no possibility of signed overflow. In this particular case, an overflow condition is present and the error_handler() method is invoked.
signed int result;
signed int si1 = INT_MIN;
if (si1== INT_MIN) error_handler("OVERFLOW ERROR", NULL, EOVERFLOW);
return -si1;
Shift Operator
The shift operator in C is between two operands of integer type.
Non-compliant Code Example
The following code will result in an unsigned overflow during the shift operation of the unsigned operands ui1 and ui2. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner which could lead to an exploitable vulnerability.
unsigned int result; unsigned int ui1 = UINT_MAX; unsigned int ui2 = sizeof(int)*8; result = ui1 << ui2;
Compliant Solution
The following compliant solution tests the suspect shift operation to guarantee there is no possibility of unsigned overflow. In this particular case, an overflow condition is present and the error_handler() method is invoked.
unsigned int result;
unsigned int ui1 = UINT_MAX;
unsigned int ui2 = sizeof(int)*8;
if( (ui2 < 0) || (ui2 >= sizeof(int)*8))
error_handler("OVERFLOW ERROR", NULL, EINVAL);
return ui1 << ui2;