There is a problem in agp code - kernel memory exhaustion (AGPIOC_RESERVE and
AGPIOC_ALLOCATE ioctls).  It is not checked whether requested pid is a pid of
the caller (no check in agpioc_reserve_wrap()). Each allocation is limited to
16KB, though, there is no per-process limit. This might lead to OOM situation,
which is not even solved in case of the caller death by OOM killer - the memory
is allocated for another (faked) process.

Reference:
https://lkml.org/lkml/2011/4/14/294

Acknowledgements:                                                                
 
Red Hat would like to thank Vasiliy Kulikov of Openwall for reporting this issue.