Bug 2459558 - libaom is not compatible with Shadow Stack protection
Summary: libaom is not compatible with Shadow Stack protection
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: aom
Version: 45
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Multimedia SIG
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 2458434
TreeView+ depends on / blocked
 
Reported: 2026-04-19 21:22 UTC by Arjun Shankar
Modified: 2026-08-17 14:34 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:


Attachments (Terms of Use)
test-CET-fix.patch (9.99 KB, text/plain)
2026-04-19 21:23 UTC, Arjun Shankar
no flags Details

Description Arjun Shankar 2026-04-19 21:22:06 UTC
libaom is built without shadow stack support due to hand written assembly files missing annotations. Therefore, Fedora applications that link against it are lacking protection at runtime.

Upstream aom git doesn't have a fix. It looks like upstream also has a CLA process for contributions.

Since the changes are mechanical (annotate function entry points with endbr64 for IBT support and add an ELF note marking CET support), I used an LLM to try to produce a downstream patch (which also applies to upstream git). I'm going to attach it to this bug. It appears to lead to a clean (but untested) scratch build on rawhide.

Some background on why this bug is being filed and marked with a "High" severity:

We (the glibc team) are planning to enable Shadow Stack protection by default in Fedora 45, and this bug report is a result of early testing. Here is a WIP/draft Fedora System-Wide Change Proposal: https://fedoraproject.org/wiki/Changes/ShadowStack

There are some notes in the Documentation section of the change proposal on how to identify and fix smaller issues (such as a stray assembly file without annotations):
https://fedoraproject.org/wiki/Changes/ShadowStack#Documentation

Reproducible: Always

Comment 1 Arjun Shankar 2026-04-19 21:23:15 UTC
Created attachment 2137622 [details]
test-CET-fix.patch

Comment 2 Fabio Valentini 2026-04-20 16:13:02 UTC
This looks like a duplicate of #2224049 ?

Comment 3 Arjun Shankar 2026-04-22 07:54:47 UTC
Thanks Fabio, it looks like that was auto closed at a Fedora release EOL, but it did lead to the filing of this upstream issue: https://aomedia.issues.chromium.org/issues/42302476

Comment 4 Aoife Moloney 2026-08-17 14:34:32 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 45 development cycle.
Changing version to 45.


Note You need to log in before you can comment on or make changes to this bug.