pg_start is copied from userspace on AGPIOC_BIND and AGPIOC_UNBIND ioctl cmds of agp_ioctl() and passed to agpioc_bind_wrap(). As said in the comment, (pg_start + mem->page_count) may wrap in case of AGPIOC_BIND, and it is not checked at all in case of AGPIOC_UNBIND. As a result, user with sufficient privileges (usually "video" group) may generate either local DoS or privilege escalation. Reference and patch: https://lkml.org/lkml/2011/4/14/293
Upstream commit: http://git.kernel.org/linus/194b3da873fd334ef183806db751473512af29ce
Statement: This issue affects the versions of Linux kernel as shipped with Red Hat Enterprise 4, 5, 6, and Red Hat Enterprise MRG. This has been addressed in Red Hat Enterprise Linux 5, 6, and Red Hat Enterprise MRG via https://rhn.redhat.com/errata/RHSA-2011-0927.html, https://rhn.redhat.com/errata/RHSA-2011-1350.html, and https://rhn.redhat.com/errata/RHSA-2011-1253.html. Red Hat Enterprise Linux 4 is now in Production 3 of the maintenance life-cycle, https://access.redhat.com/support/policy/updates/errata/, therefore the fix for this issue is not currently planned to be included in the future updates.
MITRE has assigned an additional CVE name to this issue (CVE-2011-2022), the following is the description of both flaws. Note that because the same git commit corrects both flaws, I'm not filing a second bug (as it is unlikely we would fix one and not the other). Common Vulnerabilities and Exposures assigned an identifier CVE-2011-1745 to the following vulnerability: Name: CVE-2011-1745 URL: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-1745 Assigned: 20110419 Reference: https://lkml.org/lkml/2011/4/14/293 Reference: http://openwall.com/lists/oss-security/2011/04/21/4 Reference: http://openwall.com/lists/oss-security/2011/04/22/7 Reference: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=194b3da873fd334ef183806db751473512af29ce Reference: http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.38.5 Reference: https://bugzilla.redhat.com/show_bug.cgi?id=698996 Integer overflow in the agp_generic_insert_memory function in drivers/char/agp/generic.c in the Linux kernel before 2.6.38.5 allows local users to gain privileges or cause a denial of service (system crash) via a crafted AGPIOC_BIND agp_ioctl ioctl call. Common Vulnerabilities and Exposures assigned an identifier CVE-2011-2022 to the following vulnerability: Name: CVE-2011-2022 URL: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-2022 Assigned: 20110509 Reference: https://lkml.org/lkml/2011/4/14/293 Reference: http://openwall.com/lists/oss-security/2011/04/21/4 Reference: http://openwall.com/lists/oss-security/2011/04/22/7 Reference: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=194b3da873fd334ef183806db751473512af29ce Reference: http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.38.5 Reference: https://bugzilla.redhat.com/show_bug.cgi?id=698996 The agp_generic_remove_memory function in drivers/char/agp/generic.c in the Linux kernel before 2.6.38.5 does not validate a certain start parameter, which allows local users to gain privileges or cause a denial of service (system crash) via a crafted AGPIOC_UNBIND agp_ioctl ioctl call, a different vulnerability than CVE-2011-1745.
This issue has been addressed in following products: Red Hat Enterprise Linux 5 Via RHSA-2011:0927 https://rhn.redhat.com/errata/RHSA-2011-0927.html
This issue has been addressed in following products: MRG for RHEL-6 v.2 Via RHSA-2011:1253 https://rhn.redhat.com/errata/RHSA-2011-1253.html
Acknowledgements: Red Hat would like to thank Vasiliy Kulikov of Openwall for reporting this issue.
This issue has been addressed in following products: Red Hat Enterprise Linux 6 Via RHSA-2011:1350 https://rhn.redhat.com/errata/RHSA-2011-1350.html