The normal mmap paths all avoid creating a mapping where the pgoff inside the mapping could wrap around due to overflow. However, an expanding mremap() can take such a non-wrapping mapping and make it bigger and cause a wrapping condition. There is also another case where we expand mappings hiding in plain sight: the automatic stack expansion. The wrapping condition can cause a BUG_ON() due to terminally confusing the vma_prio_tree code. Upstream patches: 982134ba62618c2d69fbbbd166d0a11ee3b7e3d8 mremap a626ca6a656450e9f4df91d0dda238fff23285f4 stack expansion downwards 42c36f63ac1366ab0ecc2d5717821362c259f517 stack expansion upwards References: http://www.spinics.net/lists/stable-commits/msg11385.html http://www.spinics.net/lists/linux-mm/msg17093.html http://groups.google.com/group/fa.linux.kernel/msg/9e43ab898c5e6d16 Acknowledgements: Red Hat would like to thank Robert Swiecki for reporting this issue.
*** Bug 713614 has been marked as a duplicate of this bug. ***
This was assigned the name CVE-2011-2496: http://seclists.org/oss-sec/2011/q2/689
This issue has been addressed in following products: MRG for RHEL-6 v.2 Via RHSA-2011:1253 https://rhn.redhat.com/errata/RHSA-2011-1253.html
This issue has been addressed in following products: Red Hat Enterprise Linux 6 Via RHSA-2011:1350 https://rhn.redhat.com/errata/RHSA-2011-1350.html
This issue has been addressed in following products: Red Hat Enterprise Linux 5 Via RHSA-2011:1386 https://rhn.redhat.com/errata/RHSA-2011-1386.html