Bug 2459561

Summary:	libvmaf is not compatible with Shadow Stack protection
Product:	[Fedora] Fedora	Reporter:	Arjun Shankar <ashankar>
Component:	vmaf	Assignee:	Nicolas Chauvet (kwizart) <kwizart>
Status:	CLOSED CANTFIX	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	kwizart
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	---
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2026-04-20 16:48:29 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2458434

Description Arjun Shankar 2026-04-19 22:07:24 UTC

libvmaf is built without shadow stack support due to hand written assembly files missing annotations. Therefore, Fedora applications that link against it are lacking protection at runtime.

Upstream vmaf git doesn't have a fix AFAIK.

Since the changes are mechanical (annotate function entry points with endbr64 for IBT support and add an ELF note marking CET support), I used an LLM to try to produce a downstream patch (which also applies to upstream git). I'm going to attach it to this bug. It appears to lead to a clean (but untested) scratch build on rawhide.

Some background on why this bug is being filed and marked with a "High" severity:

We (the glibc team) are planning to enable Shadow Stack protection by default in Fedora 45, and this bug report is a result of early testing. Here is a WIP/draft Fedora System-Wide Change Proposal: https://fedoraproject.org/wiki/Changes/ShadowStack

There are some notes in the Documentation section of the change proposal on how to identify and fix smaller issues (such as a stray assembly file without annotations):
https://fedoraproject.org/wiki/Changes/ShadowStack#Documentation

Reproducible: Always

Comment 1 Arjun Shankar 2026-04-19 22:08:15 UTC

Created attachment 2137630 [details]
test-CET-fix.patch

Comment 2 Nicolas Chauvet (kwizart) 2026-04-20 07:39:46 UTC

Thanks for the report and notice.
Please send the patch to upstream directly, I will have no mean to review it.

Comment 3 Nicolas Chauvet (kwizart) 2026-04-20 16:48:29 UTC

I cannot read the patch, but either way please send upstream so it get reviewed.