Fedora Account System
Red Hat Associate
Red Hat Customer
Spec URL: https://catanzaro.fedorapeople.org/mozilla-openh264.spec SRPM URL: https://catanzaro.fedorapeople.org/mozilla-openh264-2.4.1-1.fc41.src.rpm Description: H.264 codec support for Firefox Fedora Account System Username: catanzaro
When this package is approved, we'll delete the mozilla-openh264 subpackage from our current openh264 package. Note the current version in Cisco's repo is 2.4.1-3, so after initializing the dist-git repo, I will need to use "autorelease -b" to get ahead of the previous revision.
Hm, we have a problem. My Firefox is able to play https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/1080/Big_Buck_Bunny_1080_10s_1MB.mp4 when mozilla-openh264 is *not* installed. If I uninstall openh264, then the video fails to play. I checked a second test video as well. Conclusion: the mozilla-openh264 package is not actually doing anything and this package is useless. Firefox is not actually using the GMP plugin at all? I guess it's not really a problem per se, but we definitely need to understand what is happening.
Guess: Firefox is playing the videos using OpenH264 via ffmpeg, instead of using the OpenH264 plugin, and we can simply delete mozilla-openh264 and move on?
Hm, Neal is telling me that my tests are invalid because the GMP plugin is only used for WebRTC and not for normal web video. I tried a few websites and concluded that I don't know how to test this.
(In reply to Michael Catanzaro from comment #3) > Guess: Firefox is playing the videos using OpenH264 via ffmpeg, instead of > using the OpenH264 plugin, and we can simply delete mozilla-openh264 and > move on? I've just removed mozilla-openh264 and the video mentioned above is still playable. I guess we just need to get rid of this package.
(In reply to Michael Catanzaro from comment #4) > Hm, Neal is telling me that my tests are invalid because the GMP plugin is > only used for WebRTC and not for normal web video. > > I tried a few websites and concluded that I don't know how to test this. Ahhh snap. We need to test it a bit more then. I am going to browse YouTube, Facebook stories and Telegram videos w/o this plugin for a while and see what's happens.
(In reply to Peter Lemenkov from comment #6) > Ahhh snap. We need to test it a bit more then. I am going to browse YouTube, > Facebook stories and Telegram videos w/o this plugin for a while and see > what's happens. There's no point. It's definitely not being used for HTML video. It must be going through ffmpeg -> OpenH264. We need to figure out some way to test this with WebRTC (conference calling) while ensuring that we're not testing any other codec. E.g. a before/after to demonstrate that Webex or some other website works with the plugin installed and fails without it.
Yes, the GMP plugin is not needed for video playback at all. Video playback in firefox is done through ffmpeg instead, which in turn uses libopenh264 (the regular library, _not_ the GMP plugin). As far as I know, the GMP plugin is only used for video conferencing. See https://wiki.mozilla.org/OpenH264_plugin/Test_Plan for how to test it, in particular the "Test page" link there.
yes, GMP is not recently used for media playback.
I'm not smart enough to figure out how to use that test page. Using the presumably-good version of mozilla-openh264 from the Cisco repo, it prints a bunch of stuff, but I don't see any video.
Honestly, if this is used only for WebRTC, we could just stop disabling the upstream OpenH264 plugin and let Firefox use that for H.264? Then we wouldn't need this package at all anymore?
We need this for non-x86 arches, especially aarch64 and riscv64.
And I would prefer to use something from us across the board anyway.
I'm not sure we really need it if nobody knows how to verify that it works. :D Just kidding. OK. Well, the package is ready to review. Bonus points if somebody can figure out how to test it.
You can disable libopenh264/ffmpeg by media.ffmpeg.allow-openh264 pref at about:support. Then you may enable mozilla-openh264/GMP by media.gmp.* pref (not sure which one). Testing H.264/WebRTC is tricky, you may try https://mozilla.github.io/webrtc-landing/pc_test.html which enables to force H264 but I haven't seen H264 used by WebRTC yet, usually VP8 is used in my case.
(In reply to Martin Stransky from comment #15) > You can disable libopenh264/ffmpeg by media.ffmpeg.allow-openh264 pref at > about:support. Then you may enable mozilla-openh264/GMP by media.gmp.* pref > (not sure which one). I guessed correctly: it's media.gmp.decoder.preferred With media.ffmpeg.allow-openh264 turned off, then the test video https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/1080/Big_Buck_Bunny_1080_10s_1MB.mp4 plays if and only if media.gmp.decoder.preferred is enabled AND mozilla-openh264 from the Cisco repo is installed. If I install my new mozilla-openh264 package instead of the existing one, then it crashes. So the test definitely works :) but the new mozilla-openh264 package certainly does not, alas. The crash is in mozilla::gmp::GMPChild::GetUTF8LibPath, here: GMP_PATH_CRASH("Plugin file does not exist"); so must be a packaging issue. Investigating... > Testing H.264/WebRTC is tricky, you may try > https://mozilla.github.io/webrtc-landing/pc_test.html which enables to force > H264 but I haven't seen H264 used by WebRTC yet, usually VP8 is used in my > case. I'm going to give up on this. I'll be happy if we can get Big Buck Bunny to not crash.
Problem is I'm installing only the unversioned .so. Need to add library versioning.
(In reply to Michael Catanzaro from comment #17) > Problem is I'm installing only the unversioned .so. Need to add library > versioning. After fixing this, I wind up with a different crash: (gdb) bt #0 0x00007f5a26935c68 in MOZ_Crash () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/objdir/dist/include/mozilla/Assertions.h:317 #1 mozilla::gmp::GMPLoader::Load () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/dom/media/gmp/GMPLoader.cpp:121 #2 mozilla::gmp::GMPChild::RecvStartPlugin () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/dom/media/gmp/GMPChild.cpp:544 #3 0x00007f5a2695d3a3 in mozilla::gmp::PGMPChild::OnMessageReceived () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/objdir/ipc/ipdl/PGMPChild.cpp:859 #4 0x00007f5a2795b68b in mozilla::ipc::MessageChannel::DispatchSyncMessage () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/glue/MessageChannel.cpp:1787 #5 mozilla::ipc::MessageChannel::DispatchMessage () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/glue/MessageChannel.cpp:1737 #6 0x00007f5a27e1ed0e in mozilla::ipc::MessageChannel::RunMessage () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/glue/MessageChannel.cpp:1530 #7 mozilla::ipc::MessageChannel::MessageTask::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/glue/MessageChannel.cpp:1630 #8 0x00007f5a2795a6a4 in mozilla::RunnableTask::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/TaskController.cpp:580 #9 mozilla::TaskController::DoExecuteNextTaskOnlyMainThreadInternal () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/TaskController.cpp:907 #10 0x00007f5a27959f2e in mozilla::TaskController::ExecuteNextTaskOnlyMainThreadInternal () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/TaskController.cpp:730 #11 mozilla::TaskController::ProcessPendingMTTask () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/TaskController.cpp:516 #12 0x00007f5a27958948 in operator() () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/TaskController.cpp:234 #13 Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/nsThreadUtils.h:548 #14 nsThread::ProcessNextEvent () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/nsThread.cpp:1204 #15 0x00007f5a27e1da62 in NS_ProcessNextEvent () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/xpcom/threads/nsThreadUtils.cpp:480 #16 mozilla::ipc::MessagePump::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/glue/MessagePump.cpp:85 #17 0x00007f5a27db0c18 in MessageLoop::RunInternal () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/chromium/src/base/message_loop.cc:370 #18 MessageLoop::RunHandler () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/chromium/src/base/message_loop.cc:363 #19 MessageLoop::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/chromium/src/base/message_loop.cc:345 #20 0x00007f5a27e1d95d in nsBaseAppShell::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/widget/nsBaseAppShell.cpp:148 #21 0x00007f5a27e1d7f4 in nsAppShell::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/widget/gtk/nsAppShell.cpp:469 #22 0x00007f5a27e1c86b in XRE_RunAppShell () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/toolkit/xre/nsEmbedFunctions.cpp:712 #23 mozilla::ipc::MessagePumpForChildProcess::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/glue/MessagePump.cpp:235 #24 0x00007f5a27db0c18 in MessageLoop::RunInternal () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/chromium/src/base/message_loop.cc:370 #25 MessageLoop::RunHandler () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/chromium/src/base/message_loop.cc:363 #26 MessageLoop::Run () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/ipc/chromium/src/base/message_loop.cc:345 #27 0x00007f5a27da30d8 in XRE_InitChildProcess () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/toolkit/xre/nsEmbedFunctions.cpp:647 #28 0x0000564f7f370638 in content_process_main () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/browser/app/../../ipc/contentproc/plugin-container.cpp:57 #29 main () at /usr/src/debug/firefox-128.0.3-1.fc40.x86_64/browser/app/nsBrowserApp.cpp:378 Will attempt to debug this next. Too bad Firefox uses -g1. :(
(In reply to Michael Catanzaro from comment #17) > Problem is I'm installing only the unversioned .so. Need to add library > versioning. Actually the problem turned out to be the plugin was misnamed. The library versioning that exists today is unnecessary and not useful.
The crash is here: PRLibrary* lib = PR_LoadLibraryWithFlags(libSpec, 0); if (!lib) { MOZ_CRASH_UNSAFE_PRINTF("Cannot load plugin as library %d %d", PR_GetError(), PR_GetOSError()); Before it crashes, I see the following: [102939] Sandbox: attempt to open unexpected file /usr/lib64/firefox/libopenh264.so.7 [102939] Sandbox: attempt to open unexpected file /lib64/libopenh264.so.7 [102939] Sandbox: attempt to open unexpected file /lib64/glibc-hwcaps/x86-64-v4/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [102939] Sandbox: attempt to open unexpected file /lib64/glibc-hwcaps/x86-64-v3/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [102939] Sandbox: attempt to open unexpected file /lib64/glibc-hwcaps/x86-64-v2/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [102939] Sandbox: attempt to open unexpected file /lib64/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [102939] Sandbox: attempt to open unexpected file /usr/lib64/glibc-hwcaps/x86-64-v4/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [102939] Sandbox: attempt to open unexpected file /usr/lib64/glibc-hwcaps/x86-64-v3/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [102939] Sandbox: attempt to open unexpected file /usr/lib64/glibc-hwcaps/x86-64-v2/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [102939] Sandbox: attempt to open unexpected file /usr/lib64/libopenh264.so.7 [102939] Sandbox: seccomp sandbox violation: pid 102939, tid 102939, syscall 262, args 4294967196 140732297854400 140732297854592 0 140732297854935 0. [Child 102856, MediaDecoderStateMachine #1] WARNING: Decoder=7f4203e62000 Decode error: NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005): file /builddir/build/BUILD/firefox-128.0.3/dom/media/MediaDecoderStateMachineBase.cpp:167 [Parent 102652, IPC I/O Parent] WARNING: process 102939 exited on signal 11: file /builddir/build/BUILD/firefox-128.0.3/ipc/chromium/src/base/process_util_posix.cc:335 None of these sandbox violations occur when using the existing mozilla-openh264 package. When using the existing mozilla-openh264, the video plays fine and I see a different set of sandbox violations instead: [103496] Sandbox: attempt to open unexpected file /sys/devices/system/cpu/cpu0/cache/index2/size [103496] Sandbox: attempt to open unexpected file /sys/devices/system/cpu/cpu0/cache/index3/size [103496] Sandbox: attempt to open unexpected file /sys/devices/system/cpu/present [103496] Sandbox: attempt to open unexpected file /sys/devices/system/cpu/possible [103496] Sandbox: attempt to open unexpected file /sys/devices/system/cpu/possible [103496] Sandbox: attempt to open unexpected file /sys/devices/system/cpu/cpu0/topology/core_cpus [103496] Sandbox: attempt to open unexpected file /sys/devices/system/cpu/cpu0/topology/thread_siblings I don't know what the difference is yet.
OK, the new mozilla-openh264 works perfectly fine if I run with MOZ_DISABLE_GMP_SANDBOX=1. So that's good. But unfortunately I still don't know what is wrong.
Light bulb: the new plugin is 40 kB but the old one is 1.1 MB. Surely the new plugin is dynamically linking to openh264 but the old one is statically linking.
Well, it's impossible to static link. So we have to either (a) modify the Firefox sandbox to allow this, or (b) give up and expect users to install this package manually if they want H.264 to work in WebRTC. I wonder how hard it would be to modify the sandbox. (At any rate, this package is clearly not ready for Fedora package review. Wasn't expecting so much trouble.)
https://wiki.mozilla.org/Security/Sandbox indicates the GMP sandbox is intentionally super locked down. I'm losing faith in this plan. :) What if we try rich dependencies instead: firefox could: Requires: mozilla-openh264 if openh264 We already know how to automatically install openh264. This would probably work? We know Recommends doesn't work, but Requires probably would? Then we don't need any new package?
(In reply to Michael Catanzaro from comment #21) > OK, the new mozilla-openh264 works perfectly fine if I run with > MOZ_DISABLE_GMP_SANDBOX=1. So that's good. Yes, GMP sandbox has to be enabled and we can't modify it. We can modify Firefox req to install mozilla-openh264 if necessary, but I don't understand the change here. I see I have already installed mozilla-openh264 on my box (mozilla-openh264-2.4.1-2.fc40.x86_64) so what's the difference here? Do I understand correctly that the new mozilla-openh264 is just a wrapper over system installed cisco openh264 which is already used by ffmpeg? Thanks.
The only goal here is "Automatically install the OpenH264 codecs" https://pagure.io/fedora-workstation/issue/84 Once mozilla-openh264 is installed, there is no problem. The problem is figuring out how to make sure it gets installed automatically, without the user having to install it manually. We have previously tried: Recommends: mozilla-openh264 but this did not work. So our next plan was to build mozilla-openh264 against noopenh264, so we can install mozilla-openh264 by default instead of relegating it to the separate Cisco repository. All of this is unnecessary if we have some other way to install it. I think this should work Requires: mozilla-openh264 if openh264 We could probably have closed this bug 5.5 years ago had we considered this.... Anyway, can you try adding this Requires to the Firefox package? If it works, then we can close this package review request and don't have to figure out how to relax the sandbox rules. (In reply to Martin Stransky from comment #25) > Yes, GMP sandbox has to be enabled and we can't modify it. I certainly won't propose disabling the sandbox, but blocking the plugin from loading shared libraries is excessively strict. There's almost no security benefit to that, anyway.
(In reply to Michael Catanzaro from comment #26) > The only goal here is "Automatically install the OpenH264 codecs" > https://pagure.io/fedora-workstation/issue/84 > > Once mozilla-openh264 is installed, there is no problem. The problem is > figuring out how to make sure it gets installed automatically, without the > user having to install it manually. > > We have previously tried: > > Recommends: mozilla-openh264 > > but this did not work. So our next plan was to build mozilla-openh264 > against noopenh264, so we can install mozilla-openh264 by default instead of > relegating it to the separate Cisco repository. All of this is unnecessary > if we have some other way to install it. I think this should work > > Requires: mozilla-openh264 if openh264 > > We could probably have closed this bug 5.5 years ago had we considered > this.... Feel free to add it there by yourself, the package is open for provenpackagers. Please commit to rawhinde only.
I'm not a provenpackager, but I can open a pull request.
(In reply to Michael Catanzaro from comment #24) > https://wiki.mozilla.org/Security/Sandbox indicates the GMP sandbox is > intentionally super locked down. I'm losing faith in this plan. :) > > What if we try rich dependencies instead: firefox could: > > Requires: mozilla-openh264 if openh264 > > We already know how to automatically install openh264. This would probably > work? We know Recommends doesn't work, but Requires probably would? Then we > don't need any new package? It should be okay, we probably need to account for this for Fedora Asahi Remix... Hector and Davide: we should be able to handle this, right?
Why can't we just patch firefox to add the openh264 lib to the sandbox? https://searchfox.org/mozilla-central/source/security/sandbox/linux/Sandbox.cpp#730 The file list is right there. This seems like a reasonable use case. This isn't the first time we had to get browser sandboxes modified to fix stuff (e.g. we had it with Mesa driver stuff in Chrome IIRC). More generally: This is arguably a bug in Firefox. The plugin .so links to libopenh264 using standard dynamic linking (it doesn't try to dlopen() it at runtime or anything). That means Firefox has the capacity to inspect the plugin file before loading it and add its required libraries to the allowed files list. In fact it requires other libs: # ldd module/libgmp-openh264.so linux-vdso.so.1 (0x0000ffffbb150000) libopenh264.so.7 => /lib64/libopenh264.so.7 (0x0000ffffbb070000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000ffffbae00000) libm.so.6 => /lib64/libm.so.6 (0x0000ffffbad50000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000ffffbacf0000) libc.so.6 => /lib64/libc.so.6 (0x0000ffffbab10000) /lib/ld-linux-aarch64.so.1 (0x0000ffffbb108000) I suspect the only reason this worked at all until now is because Firefox already loads all of those other libraries (libstdc++.so.6, libm, etc.) so the dynamic linker doesn't have to open the files again when loading the plugin. Re FAR, I think we just need to make sure all of our extrafiles are installed in one dnf invocation. As long as openh264 and mozilla-openh264 are both extrafiles and installed together, I don't think dnf will complain.
(In reply to Hector Martin from comment #30) > Why can't we just patch firefox to add the openh264 lib to the sandbox? Well if the Requires strategy works, then there's no point. Maintaining the plugin as a separate source package is going to be work. Sure seems like there is no need for it. I will delete https://pagure.io/fedora-workstation/mozilla-openh264 if the Requires works, as I expect.
It works: ============================================================================================================================================================= Package Architecture Version Repository Size ============================================================================================================================================================= Upgrading: firefox x86_64 129.0-2.fc42 @commandline 70 M firefox-langpacks x86_64 129.0-2.fc42 @commandline 47 M nss x86_64 3.103.0-1.fc40 updates 710 k nss-softokn x86_64 3.103.0-1.fc40 updates 412 k nss-softokn-freebl x86_64 3.103.0-1.fc40 updates 309 k nss-sysinit x86_64 3.103.0-1.fc40 updates 19 k nss-util x86_64 3.103.0-1.fc40 updates 89 k Installing dependencies: mozilla-openh264 x86_64 2.4.1-2.fc40 fedora-cisco-openh264 428 k This was all a waste of time. Closing.
Nice, I think the rich requires are a good solution for this! Or at least for the rpm build - for the flatpak build, I don't think they are going to help at all. For firefox flatpak I think the original solution you had going here would be more helpful as we would only need to use a single extension point to swap out noopenh264 with openh264 then.
(In reply to Kalev Lember from comment #33) > Nice, I think the rich requires are a good solution for this! Or at least > for the rpm build - for the flatpak build, I don't think they are going to > help at all. For firefox flatpak I think the original solution you had going > here would be more helpful as we would only need to use a single extension > point to swap out noopenh264 with openh264 then. Do we have a flatpak extension for openh264, though? I don't mind bringing this back if it would be useful.
We don't have the extension yet, but it's planned - similar to how the freedesktop openh264 extension works using the extra-data mechanism.
I have attached an archive of the git repo here: https://pagure.io/fedora-workstation/issue/84#comment-925311 It would only be useful if we can sort out the hyper-aggressive sandboxing issue. (Of course, by the time somebody wants to use this in the future, there will likely be a newer version of OpenH264 and you will have to update the files from latest upstream sources. See the README.)