Bug 2577704 (CVE-2026-99738)

Summary: CVE-2026-99738 kernel: kernel: Denial of service via infinite retry loop during shared memory swapin
Product: [Other] Security Response Reporter: OSIDB Bzimport <bzimport>
Component: vulnerabilityAssignee: Product Security DevOps Team <prodsec-dev>
Status: NEW --- QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: unspecifiedCC: akhatavk, aos-team-art-private, asdas, dpaolell, jdelft, jupierce, lgarciaa, mbiarnes, ppalepu, ppostler, prdhamdh, rhel-process-autobot, sghai, sidsharm, suppawar, vlaad, watson-tool-maintainers
Target Milestone: ---Keywords: Security
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: ---
Doc Text:
A flaw was found in the Linux kernel's shared memory (shmem) subsystem. When swapping in large memory entries, an allocation failure mishandles internal tracking values, incorrectly converting an out-of-memory error into an entry-already-exists error. Because the kernel continuously retries operations upon receiving this error code, persistent allocation failures cause the affected task to spin indefinitely. A local user can exploit this behavior to cause high CPU consumption and a denial of service (DoS).
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description OSIDB Bzimport 2026-11-20 06:09:15 UTC
In the Linux kernel, the following vulnerability has been resolved:

mm/shmem: preserve allocation errors when swapping in large entries

shmem_swapin_folio() adjusts swap to the requested slot within a large
swap entry before allocating a folio. If allocation or charging fails,
the error path compares this adjusted value with the original entry
still stored in the XArray in shmem_confirm_swap().

Unless the index is at the start of the entry, this turns -ENOMEM into
-EEXIST even though nothing changed.

The caller retries -EEXIST, so persistent allocation failures can leave
the task spinning instead of returning the error.

Restore swap to index_entry on allocation failure so the existing error
path checks the original entry. This preserves the allocation error if
the entry is unchanged and still retries if it changed concurrently.