Bug 2459561 - libvmaf is not compatible with Shadow Stack protection
Summary: libvmaf is not compatible with Shadow Stack protection
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: vmaf
Version: rawhide
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Nicolas Chauvet (kwizart)
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 2458434
TreeView+ depends on / blocked
 
Reported: 2026-04-19 22:07 UTC by Arjun Shankar
Modified: 2026-04-20 16:48 UTC (History)
1 user (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2026-04-20 16:48:29 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Arjun Shankar 2026-04-19 22:07:24 UTC
libvmaf is built without shadow stack support due to hand written assembly files missing annotations. Therefore, Fedora applications that link against it are lacking protection at runtime.

Upstream vmaf git doesn't have a fix AFAIK.

Since the changes are mechanical (annotate function entry points with endbr64 for IBT support and add an ELF note marking CET support), I used an LLM to try to produce a downstream patch (which also applies to upstream git). I'm going to attach it to this bug. It appears to lead to a clean (but untested) scratch build on rawhide.

Some background on why this bug is being filed and marked with a "High" severity:

We (the glibc team) are planning to enable Shadow Stack protection by default in Fedora 45, and this bug report is a result of early testing. Here is a WIP/draft Fedora System-Wide Change Proposal: https://fedoraproject.org/wiki/Changes/ShadowStack

There are some notes in the Documentation section of the change proposal on how to identify and fix smaller issues (such as a stray assembly file without annotations):
https://fedoraproject.org/wiki/Changes/ShadowStack#Documentation

Reproducible: Always

Comment 1 Arjun Shankar 2026-04-19 22:08:15 UTC
Created attachment 2137630 [details]
test-CET-fix.patch

Comment 2 Nicolas Chauvet (kwizart) 2026-04-20 07:39:46 UTC
Thanks for the report and notice.
Please send the patch to upstream directly, I will have no mean to review it.

Comment 3 Nicolas Chauvet (kwizart) 2026-04-20 16:48:29 UTC
I cannot read the patch, but either way please send upstream so it get reviewed.


Note You need to log in before you can comment on or make changes to this bug.