CIS 4615 meeting -*- Outline -*- * Integer Overflow Attacks Based on chapter 7 of the book: 24 Deadly Sins of Software Security by M. Howard, D. LeBlanc, and J. Viega (McGraw-Hill, 2010). ** Background ------------------------------------------ INTEGER ARITHMETIC In C, C++, Java, C# is modulo arithmetic ------------------------------------------ See the file intarith.c for an example *** C conversion rules Based on https://www.securecoding.cert.org/confluence/display/ c/INT02-C.+Understand+integer+conversion+rules ------------------------------------------ C CONVERSION RULES Integer promotions: numbers smaller than int are promoted: to int, if possible to unsigned int, otherwise char c1 = 200; char c2 = 56; char c = c1 + c2; Integer Conversion Rank lattice (C99): long long int unsigned long long int | | long int unsigned long int | | int unsigned int | / \ | short int unsigned short int | \ / | signed char char unsigned char \ | / _Bool ------------------------------------------ Q: Why would C do integer promotions? To avoid arithmetic errors in intermediate values Q: What is the value of c? 0, 256 is truncated to the size of a char. In the lattice height is same as higher rank. ------------------------------------------ Usual Arithmetic Conversions: Goal: balance argument types 1. Stop after integer promotion, if balanced 2. If both are signed (or unsigned), convert the argument with lower integer rank to the higher integer rank 3. If the arg with higher rank is unsigned, convert the other arg to that type 4. Otherwise, convert the other arg to the higher signed type ------------------------------------------ Usual arithmetic conversions also balance the 2nd and 3rd args of the ?: operator ------------------------------------------ EXAMPLE What does this C code print? int si = -1; unsigned int ui = 1; printf("%d\n", si < ui); What can happen in this code? const long int MAXLEN = 0x7fff; short len = strlen(input); if (len < MAXLEN) { printf("sad\n"); } else { printf("happy\n"); } ------------------------------------------ ... 0, because si is converted to unsigned int, and becomes UINT_MAX Q: What answers can the second code give? both "sad" and "happy" "sad" happens because len is promoted to a long, if it was negative (due to wrap around), then it is extended to be negative and will be less than MAXLEN, which is positive. For example, if len is 0xffff;, then it becomes the largest negative int, 0xffffffff; Moral: you have to be careful... *** gotchas ------------------------------------------ GOTCHAS What does this print? signed char c = -128; signed char result = c/-1; printf("result = %d\n", result); Comparisons between signed and unsigned ints Binary operators on signed ints: int flags = 0x7f; char LowByte = 0x80; if ((char)flags ^ LowByte == 0xff) { return SUCCESS; } return FAILURE; ------------------------------------------ ... -128, because 128 can't be represented as ... FAILURE because both operands of ^ are promoted to int, so flags is sign extended to 0x0000007f and LowByte is extended to 0xffffff80, so the result is 0xffffffff, which is not 0x000000ff! *** other languages ------------------------------------------ OTHER LANGUAGES Java - has only signed integers built in - rules are just like C (on purpose) - no exceptions for overflows C# - rules are like Java (and thus C) but will use 64 bit integers - has stricter type checking than C byte a=1, b=255; byte c = a+b; // type error! - has checked code blocks, throws error on overflow Perl - uses double precision floats to represent all numbers ------------------------------------------ ** attack *** steps ------------------------------------------ ATTACK 1. Find a calculation that 2. Feed the program inputs that 3. Use that to ------------------------------------------ ... can be surprising (incorrect) and is used in some security-critical way: - buffer size calculation - array indexing ... cause the size calculation to be surprising ... perform some other attack: - buffer overflow - writing other parts of memory - crash the program Q: Where is the user input used in this attack? To force the calculation to exhibit an error *** implications ------------------------------------------ IMPLICATIONS Mac OS X 10.4.11 - denial of service from integer overflows Android SDK: special BMP file with negative offset field leads to buffer overflow attack Microsoft IIS server: HTR handler accepted length of 64K-1, then added 1, and allocated 0 bytes ------------------------------------------ ... (IIS) leading to widespread buffer overflow attacks Moral: incorrect calculations can lead to exploits! ** remediation ------------------------------------------ REMEDIATION 1. Do the math ? Size = (elements * sizeof(element)) + sizeof(header) How many elements can be handled? 2. Temporarily write out casts to help unsigned int x; short a, b; /* ... */ if (a+b < x) { DoSomething(); } 3. Don't use "clever tricks" int a, b, c; c = a*b; /* check for overflows */ if (c < 0) { return OVERFLOW; } ------------------------------------------ Problem if: MAX_INT <= (elements * sizeof(element)) + sizeof(header) iff {subtracting sizeof(header) from both sides} MAX_INT - sizeof(header) <= (elements * sizeof(element)) iff {dividing by sizeof(element)} (MAX_INT - sizeof(header)) / sizeof(element) <= elements the left hand side is a compile-time constant, which can be used in comparisons with elements. ...write code that is straightforward and easy to understand Q: What would be the casts in the code for 2? integer promotion on a+b to int, then Promotion to unsigned int because x has higher rank ((unsigned int) (int) (a+b)) < x but that could cause trouble if the intermediate result is negative. Q: What's wrong with the code in 3? It only checks for some overflows: if a == 1<<30+1, and b== 8, then the result is 2^33+8, which is 8 when truncated back to 32 bits, and that is not negative. Q: How could you fix that? Use a bigger type to store the result. ------------------------------------------ OTHER DEFENSES Use SafeInt package www.codeplex.com/SafeInt In gcc can catch signed integer overflows using -ftrapv Static analysis tools ------------------------------------------