From Adam Litke's mail to LKML (http://lkml.org/lkml/2007/1/29/180): [PATCH] Don't allow the stack to grow into hugetlb reserved regions When expanding the stack, we don't currently check if the VMA will cross into an area of the address space that is reserved for hugetlb pages. Subsequent faults on the expanded portion of such a VMA will confuse the low-level MMU code, resulting in an OOPS. Check for this.
see bz#253313 for more details
This CVE also affects RHEL-3 (advisory has not been filled yet). On the ppc there is in mm/mmap.c file function called "expand_stack" which has the same function as that one mentioned in the patch. This also checks first, if there is enough memory to allocate, then checks if the stack wasn't expanded by some other thread and if not, checks the address space limits. If none from these tests passes, the stack is expanded. But before this increase one yet check should be added -- if the address space we would like to allocate doesn't belong to the huge_tlb_only space.