Description of problem: This problem is present in two string functions of glibc: * strcoll_l * strxfrm In these functions, the code needs to choose whether to allocate its internal working buffers using the stack or the heap. It makes this decision by calling __libc_use_alloca -- if it returns true it uses ::alloca, otherwise it uses ::malloc. The problem is that the value it passes into __libc_use_alloca is not the number of bytes it will need to allocate, but rather a count of the number of *characters*. The result is that, for example, if I strcoll two strings of 5 characters each, the function will do __libc_use_alloca(10) and then subsequently proceed to allocate 5 times this number on the stack. If you have a thread stack size of 256KB and you compare 2 strings that are 30,000 characters each, the function will do __libc_use_alloca(60000), which returns true because 60,000 is under the threshold of 64KB. Then it will alloca a total of 300,000 bytes (num_chars * (sizeof(int32_t)+1)), which will overflow the stack. The functions should be changed to pass in the true number of bytes that will be allocated when calling __libc_use_alloca.
Hello Scott, I'm the Onsite Engineer from Red Hat at the SAP LinuxLab. While reviewing the Bugzilla request opened by SAP in the past I found this bug reported by you. Since this bug is already over a year old could you let us know if this is still an issue or if this does not occur any more on later versions of RHEL5? Thanks and regards, Frank Danapfel Red Hat Onsite Engineer @ SAP LinuxLab
Hi Frank, I'm not in a position at this point to reproduce the problem without putting in some effort. It's not really necessary, though, because the bug is very obvious from simply looking at the code. If you crack open one of the functions I reference above (e.g. strcoll_l) and look for the value it passes into __libc_use_alloca. If it is number of characters, the bug is still present. If it is number of bytes, the bug was fixed. Judging by the response time on this bug report that I submitted over a year ago, I'm going to take a wild guess and wager that the bug has not been fixed (and likely the surrounding code has not even been touched). Thanks, scott
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2012-0260.html