mmap_min_addr is intended to prevent a process from mmaping virtual memory below a certain point.  This is useful in hardening the kernel again subversion in the face of NULL ptr bugs.  We currently set the minimum threshold to 65535 in RHEL kernels.

The fedora infrastructure got a number of mmap_zero selinux denials.  In investigating why selinux was denying mmap to low pages we found audit syscall records like so:

type=SYSCALL arch=i386 syscall=mmap2 success=no exit=-13(Permission denied) a0=0 a1=1000 a2=3 a3=22

looking at a0-a4 we see that the program called

mmap(NULL, PAGE_SIZE, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE,...)

which is a perfectly legal call to mmap.  But SELinux denied the call with:

node=10.8.34.193 type=AVC msg=audit(06/16/2009 03:15:50.856:139394) : avc:  denied  { mmap_zero } for  pid=25660 comm=httpd scontext=user_u:system_r:httpd_t:s0 tcontext=user_u:system_r:httpd_t:s0 tclass=memprotect 

(which explains the errno=EACCESS on the mmap)

This means that the program was allowing the kernel to pick an address for the mmap location and the kernel picked an address < 64k.  Upon code review Rik van Riel pointed me to arch_get_unmapped_area_topdown() which walks down the virtual memory space looking for room to make an allocation.  It will allocate all the way down to the first page (but not the zero page)  So if the process is running low on virtual memory space, or if they have just done a LOT of mmap calls they could push into this range and get the accidental denials.

I propose as a solution for RHEL5 to change mmap_min_addr to only protect 0-4096 instead of 0-65535.  This should provide nearly the same protection (very few kernel structs and offsets are >4k but <64k) and is completely safe since the code has always protected against giving out the first 4k of space.

Upstream I am going to attempt to rewrite all of the *_get_unmapped_area_* functions to make sure they won't walk into mmap_min_addr space, but changes to such deep function in RHEL is inadvisable at best.