Escalated to Bugzilla from IssueTracker
Created attachment 135788 [details] Proposed Patch
Created attachment 136333 [details] Proposed Patch Added write locks around kfree to resolve a memory corruption issue.
Created attachment 136605 [details] Proposed Patch moved br_write_lock and unlock to ipc_unlock_deleted
Deadlock happens on smp machines when a shmat() function runs at the same time that shmctl() is used to remove the same shm. While the shmat process waits at the spinlock in ipc_lock function, the shmctl function deletes the path to the kern_ipc_perm structure which holds the spinlock variable and frees the memory location. Because of this, the ipc_unlock() run for shmctl() does not unlock the spinlock and the shmat waits indefinitely at the spinlock. This is serious issue for RHEL3 users especially applications using IPC shared-memory. This deadlock brings other deadlock in IPC features (ex. IPC semaphore) since deadlocked process holds readlock of BR_SEMAPHORE_LOCK which are shared by all IPC features. The code in question was backported from the 2.5 kernel with the patch linux-2.4.21-semaphore-scale.patch by Ingo Molnar. This code does not appear upstream in the 2.4 kernels. The Red Hat implementation on the 2.4 kernel is done using 'Big Reader' read-write locks. The patch was written by modifying the code from the patch provided in bz # 91146 to use brlocks. The attached reproducer recreates the issue.
A fix for this problem has just been committed to the RHEL3 U9 patch pool this evening (in kernel version 2.4.21-47.1.EL).
A fix for this problem has also been committed to the RHEL3 E9 patch pool this evening (in kernel version 2.4.21-47.0.1.EL).
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2006-0710.html