Bug 2065030

Summary: [Not related to RDD sandbox] VAAPI stopped working in Firefox 98
Product: [Fedora] Fedora Reporter: Akarshan Biswas <akarshan.biswas>
Component: firefoxAssignee: Gecko Maintainer <gecko-bugs-nobody>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 35CC: akarshan.biswas, ekohlvan, erack, gecko-bugs-nobody, jhorak, kai-engert-fedora, klaas, pjasicek, rhughes, rstrode, sandmann, stransky
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-04 07:08:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Requested log
none
Firefox video acceleration debug logs none

Description Akarshan Biswas 2022-03-17 07:28:31 UTC
Description of problem:
VAAPI hardware decoding stopped working in Firefox 98, 


Version-Release number of selected component (if applicable): 98


How reproducible:
Everytime

Steps to Reproduce:
1. Disable RDD sandbox
2. Play a video
3. Observe the logs and cpu usage

Actual results:
vaapi does not work. The decoder changed to software with this failure:

[RDD 4329: MediaPDecoder #3]: D/PlatformDecoderModule   failed to create texture over DMABuf memory!


Expected results:
vaapi should work as it used to in Firefox 97

Additional info:
GPU information:

00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Stoney [Radeon R2/R3/R4/R5 Graphics] (rev c1) (prog-if 00 [VGA controller])
	Subsystem: Acer Incorporated [ALI] Device 1099
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 36
	Region 0: Memory at e0000000 (64-bit, prefetchable) [size=256M]
	Region 2: Memory at f0000000 (64-bit, prefetchable) [size=8M]
	Region 4: I/O ports at 3000 [size=256]
	Region 5: Memory at f0d00000 (32-bit, non-prefetchable) [size=256K]
	Expansion ROM at 000c0000 [disabled] [size=128K]
	Capabilities: <access denied>
	Kernel driver in use: amdgpu
	Kernel modules: amdgpu

Comment 1 Martin Stransky 2022-03-17 08:16:52 UTC
Can you run firefox on terminal with:

MOZ_LOG="PlatformDecoderModule:5, Dmabuf:5" MOZ_DISABLE_RDD_SANDBOX=1

env variables, try to play the video and attach the log here?

Thanks.

Comment 2 Akarshan Biswas 2022-03-17 08:31:39 UTC
Created attachment 1866341 [details]
Requested log

Uploaded the log.
Not sure what went wrong.

Comment 3 George Abraham 2022-03-25 22:47:39 UTC
Created attachment 1868375 [details]
Firefox video acceleration debug logs

Comment 4 George Abraham 2022-03-25 22:48:57 UTC
Possibly related, see attached logs above:
After upgrading from Fedora 35 to 36, VAAPI video acceleration is broken on Firefox 98:
[Child 143892: Main Thread]: D/PlatformDecoderModule PDMInitializer, Init PDMs in Content process
[Child 143892: Main Thread]: D/PlatformDecoderModule FFMPEG: VA-API FFmpeg is disabled by platform
[Child 143892: Main Thread]: D/PlatformDecoderModule FFMPEG: VA-API FFmpeg is disabled by platform
[Child 143892: Main Thread]: D/PlatformDecoderModule FFMPEG: Couldn't load function av_frame_get_colorspace
[Child 143892: Main Thread]: D/PlatformDecoderModule FFMPEG: Couldn't load function av_frame_get_color_range
[Child 143892: Main Thread]: D/PlatformDecoderModule Sandbox RDD decoder supports requested type
[RDD 144083: Main Thread]: D/PlatformDecoderModule PDMInitializer, Init PDMs in RDD process
[RDD 144083: Main Thread]: D/PlatformDecoderModule FFMPEG: Couldn't load function av_frame_get_colorspace
[RDD 144083: Main Thread]: D/PlatformDecoderModule FFMPEG: Couldn't load function av_frame_get_color_range
[RDD 144083: Main Thread]: D/PlatformDecoderModule Agnostic decoder supports requested type

[george@fedora ~]$ vainfo 
libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_13
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.14 (libva 2.14.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 22.1.1 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileNone                   :	VAEntrypointStats
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
..
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointFEI
      VAProfileHEVCMain               :	VAEntrypointEncSliceLP
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointEncSliceLP
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile1            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileVP9Profile3            :	VAEntrypointVLD

In about:config, these are the values that I modified:
Key 	 	 	 	Value
security.sandbox.content.level 	0
gfx.webrender.all 	 	true
media.ffmpeg.vaapi.enabled 	true
media.ffvpx.enabled 	 	false
media.rdd-vpx.enabled 	 	false

Codecs used in Youtube (with enhanced-h264ify extension):
vp09.00.51.08.01.01.01.01.00 (248) / opus (251)

Firefox is started with:
MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_LOG="PlatformDecoderModule:5" MOZ_DISABLE_RDD_SANDBOX=1 MOZ_ENABLE_WAYLAND=1 /home/george/Downloads/firefox/firefox

Comment 5 Akarshan Biswas 2022-03-26 13:33:57 UTC
> FFMPEG: Couldn't load function av_frame_get_colorspace


I think this is entirely a different issue from the one I reported, probably related to FFMPEG 5. Martin has reported upstream and submitted a patch today, bug id is 1761471.

Let's wait for Martin's confirmation.

Comment 6 George Abraham 2022-03-26 15:21:06 UTC
Thanks for the info. The bug is private, I can't access it. I have found 2 related bugs below, both closed, but we'll have to wait.
https://bugzilla.mozilla.org/show_bug.cgi?id=1750760 - Implement support for ffmpeg 5.0 - target release - Firefox 99
https://bugzilla.mozilla.org/show_bug.cgi?id=1759137 - FFMPEG 5.0 Crash in [@ av_buffer_ref] - target release - Firefox 100

Comment 7 Akarshan Biswas 2022-03-26 17:27:01 UTC
I don't think that bug report is private.


I'm providing the link.

https://bugzilla.mozilla.org/show_bug.cgi?id=1761471

Comment 8 George Abraham 2022-03-26 19:33:26 UTC
Your bug is from the Firefox Bugzilla, not Redhat.
Yes, it is publicly available.
Thanks, the patch will hopefully be merged soon.

Comment 9 Akarshan Biswas 2022-04-03 06:45:13 UTC
Update:

I tested Firefox Nightly version: 100.0a1 ; Build id. 20220402213419

vaapi works.

Comment 10 Martin Stransky 2022-04-04 06:05:12 UTC
Does it really affect Fedora 35? It should be Fedora 36 only as Fedora 35 has ffmpeg 4.1 which should work fine.

These two should be true (default):

media.ffvpx.enabled 	 	false
media.rdd-vpx.enabled 	 	false

See https://fedoraproject.org/wiki/Firefox_Hardware_acceleration - it's up to date.

Comment 11 Akarshan Biswas 2022-04-04 06:14:14 UTC
Yes. It does effect in Fedora 35 with ffmpeg 4 as I am currently testing on it.

As the logs says, If I understand correctly , the EGLImageKHR fails on UID2 and after that it goes into software (ffmpeg decoder).


 D/Dmabuf DMABufSurfaceYUV::CreateEGLImage() UID 2 plane 0
 D/Dmabuf   EGLImageKHR creation failed
 D/Dmabuf DMABufSurfaceYUV::ReleaseEGLImages() UID 2
 D/PlatformDecoderModule   failed to create texture over DMABuf memory!


This only happens on Firefox 98 not in nightly. Today I will test firefox beta.

Comment 12 Martin Stransky 2022-04-04 07:18:19 UTC
Hm, that's strange, I'm not aware of any other va-api fix in 99/100.

Can you try to use mozregression to find where it was fixed?
https://fedoraproject.org/wiki/How_to_debug_Firefox_problems#Use_Mozregression_tool
(you can use "--profile /home/*user_name*/.mozilla/firefox/*profile_name* --profile-persistence clone-first" params to clone your working profile so you don't need to set all preferences for each iteration).

Comment 13 Akarshan Biswas 2022-04-04 11:57:05 UTC
I wasnt able to run it on version 100 for some reason, I ran it between 97 (working vaapi) and version 98 (regression) and got the following output:

12:47.76 INFO: No more integration revisions, bisection finished.
12:47.76 INFO: Last good revision: c8abc89e09ceaa3dcc7cb4b36ce3abb64e42a43f
12:47.76 INFO: First bad revision: 582ee6bee6e9a8915466e43b7d222193a27ed5a4
12:47.76 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=c8abc89e09ceaa3dcc7cb4b36ce3abb64e42a43f&tochange=582ee6bee6e9a8915466e43b7d222193a27ed5a4


Meanwhile tested Firefox beta 99.0b8 ; buildid: 20220324185704 and vaapi still is not working in this version.

Comment 14 Akarshan Biswas 2022-04-04 13:27:48 UTC
Correction : After testing multiple times with the mozregression tool(learning in the process), I was finally able to pinpoint(hopefully).

12:54.89 INFO: No more integration revisions, bisection finished.
12:54.89 INFO: Last good revision: d9aa83a3159f8c5368d186f86ffb41f63c460ebd
12:54.89 INFO: First bad revision: d87eb81f84958645ede935cfcc4b6dec93314668
12:54.89 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=d9aa83a3159f8c5368d186f86ffb41f63c460ebd&tochange=d87eb81f84958645ede935cfcc4b6dec93314668



Using --find-fix(didn't know about this flag earlier.) ; I was able to pinpoint the fix in Nightly

16:28.98 INFO: No more integration revisions, bisection finished.
16:28.98 INFO: First good revision: 7403fb5ad11f0c605630c1c71eccb912a4685821
16:28.98 INFO: Last bad revision: 6f139fed188c86dbb1de59f4249ce03871cdf626
16:28.98 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=6f139fed188c86dbb1de59f4249ce03871cdf626&tochange=7403fb5ad11f0c605630c1c71eccb912a4685821


Hopefully this isn't a false positive and sorry for the last one.

Comment 15 Martin Stransky 2022-04-05 08:15:24 UTC
If FF99 is still affected then https://bugzilla.mozilla.org/show_bug.cgi?id=1761832 looks like a good candidate.

Comment 16 Akarshan Biswas 2022-04-06 05:00:22 UTC
hmm. Can it be backported?

Comment 17 Martin Stransky 2022-04-06 08:23:48 UTC
Will try so.

Comment 18 Akarshan Biswas 2022-04-07 05:21:51 UTC
Thank you.

Comment 19 Martin Stransky 2022-04-07 07:50:30 UTC
Unfortunatelly it failed to apply/build. Please wait to FF 100.

Comment 20 Akarshan Biswas 2022-05-01 06:36:54 UTC
Update: I am getting occasional crashes like this in the latest nightly 101.
https://crash-stats.mozilla.org/report/index/7bebc4da-f194-430f-9e98-8a0980220430

Comment 21 Martin Stransky 2022-05-04 07:08:13 UTC
Fixed in firefox-100.0-2 builds.
https://bodhi.fedoraproject.org/updates/FEDORA-2022-2a90168c37