Created attachment 106045 [details] Patch for this issue. This patch was taken from http://www.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.23aa3/00_VM_IO-4
The issue is fairly simple in its basis. When a driver allocates resources and marks the pages PG_Private (for DMA use) the pages are no longer refcounted. Now imagine someone doing O_DIRECT dma on an audio mmap and closing the mmap. Not pretty 8) The needed fix as far as I can see for RHEL3 is to set VM_IO (ie disallow the O_DIRECT) in remap_page_range(). We can do that because unlike base 2.4 we happen to pass the vma into the function.
This is the 2.6 addition (which is like our 2.4 because we pass the vma for Riel mm) /* * Physically remapped pages are special. Tell the * rest of the world about it: * VM_IO tells people not to look at these pages * (accesses can have side effects). * VM_RESERVED tells swapout not to try to touch * this region. */ vma->vm_flags |= VM_IO | VM_RESERVED;
A fix for this problem has just been committed to the RHEL3 U7 patch pool this evening (in kernel version 2.4.21-37.2.EL).
A fix for this problem has also been committed to the RHEL3 E7 patch pool this evening (in kernel version 2.4.21-37.0.1.EL).
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2006-0140.html