Description of problem: This kernel enables unmapped area topdown search sequence at compatibility mode of EM64T, but forgets to enable macro HAVE_ARCH_HUGETLB_UNMAPPED_AREA Version-Release number of selected component (if applicable): OS Version: RHEL4-pre-rc1 kernel version: 2.6.9-1.849_EL How reproducible: Steps to Reproduce: 1. echo "100">/proc/sys/vm/nr_hugepages 2. gcc -m32 rhel4prerc1_hugtlb_mmap_fail.c 3. a.out Actual results: Segment fault happens Expected results: No segment fault happens Additional info:
Created attachment 108937 [details] test case to reproduce bug
so it seems like we need the following patch: --- linux-2.6.9/include/asm-x86_64/page.h.bak Wed Dec 22 15:33:35 2004 +++ linux-2.6.9/include/asm-x86_64/page.h Wed Dec 22 15:34:56 2004 @@ -24,10 +24,13 @@ #define LARGE_PAGE_MASK (~(LARGE_PAGE_SIZE-1)) #define LARGE_PAGE_SIZE (1UL << PMD_SHIFT) +#ifdef CONFIG_HUGETLB_PAGE #define HPAGE_SHIFT PMD_SHIFT #define HPAGE_SIZE ((1UL) << HPAGE_SHIFT) #define HPAGE_MASK (~(HPAGE_SIZE - 1)) #define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT) +#define HAVE_ARCH_HUGETLB_UNMAPPED_AREA +#endif #ifdef __KERNEL__ #ifndef __ASSEMBLY__
i'll build a test kernel with that patch
This bug still exists in RHEL4-RC
Has the above patch been tested? If not, i'll build a kernel for us to test. thanks.
Yes. It works. But we didn't see this patch has been integrated into RHEL4-RC.
If we apply the patch at comment 2, another bug will be triggered. hugetlb_get_unmapped_area_topdown in file arch/i386/mm/hugetlbpage.c might cause vma overflow out of 4GB address space of IA32 processes on x86_64. Then, the third bug will be triggered that a memory leak will happen when the IA32 process exit because kernel just releases the page table below 4GB. I will add the test case and a patch to fix these bugs.
Created attachment 110234 [details] The test case to show the unmapped area overflow beyond 4GB for IA32 processes Reproduce it: 1) Apply the patch of comment 2 to kernel; 2) #mkdir /root/test/mmap/ 3) #echo "100">/proc/sys/vm/nr_hugepages 4) #mount -t hugetlbfs none /root/test/mmap 5) #gcc -o test4 -m32 test4.c 6) #./test4 7) #cat /proc/test4_pid/maps If we delete the getchar() in test4.c and use '#while true; do ./test4; done' to execute test4 looply, the memory will be used up soon.
Created attachment 110235 [details] The patch to fix these bugs Pls. don't apply the patch of comment #2 before applying rhel4-rc-hugetlb_mmap_fail2.patch.diff. rhel4-rc-hugetlb_mmap_fail2.patch.diff includes the patch of comment #2.
Ok. thanks. I don't see this patch posted to upstream?
I guess its not upstream b/c flexmap isn't there for x86_64
This bug still exists in RHEL4
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2005-420.html