Hello, After upgrading to firefox-94.0-1, I'm getting a lot of crashes in tabs with a lot of media, such as youtube, online games or even the french post office. The crashes don't happen as soon as the pages load, but after some time. When they do happen, it is impossible to reload the tab that crashed or open a new one. Sometimes other open tabs start crashing after that first one and once I had to kill Firefox from the terminal. Every time the issue occurs, I see these two lines in the system log: Nov 07 22:54:14 host firefox.desktop[52499]: [Parent 52499, IPC I/O Parent] WARNING: SCM_RIGHTS message was truncated cmsg_len:20 fd:198: file /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:370 Nov 07 22:54:14 host firefox.desktop[59890]: Exiting due to channel error. There are also some other errors that show up in the log, before the tabs crash, that I did not get before the update, for instance I get quite a few of those: Nov 07 17:47:32 host firefox.desktop[40268]: ATTENTION: default value of option mesa_glthread overridden by environment. Nov 07 19:56:03 host firefox.desktop[10122]: IPDL protocol Error: Received an invalid file descriptor Nov 07 19:56:03 host firefox.desktop[10122]: [Parent 10122, IPC I/O Parent] WARNING: Message needs unreceived descriptors channel:7fc37370b280 message-type:65531 header()->num_fds:1 num_fds:0 fds_i:0: file /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:502 Nov 07 19:56:03 host firefox.desktop[10122]: IPDL protocol Error: Received an invalid file descriptor Nov 07 19:56:03 host firefox.desktop[10122]: [Parent 10122, IPC I/O Parent] WARNING: Unable to create pipe in server mode error(Too many open files).: file /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:156 Nov 07 19:56:03 host firefox.desktop[10122]: IPDL protocol Error: Received an invalid file descriptor Nov 07 19:56:03 host firefox.desktop[41422]: [Child 41422, MediaDecoderStateMachine #1] WARNING: Decoder=7f5d61e56c00 Decode error: NS_ERROR_OUT_OF_MEMORY (0x8007000e): file /builddir/build/BUILD/firefox-94.0/dom/media/MediaDecoderStateMachine.cpp:3562 a lot of the above repeat several times, then I get these: Nov 07 19:56:03 host firefox.desktop[10122]: IPDL protocol Error: Received an invalid file descriptor Nov 07 19:56:03 host firefox.desktop[10122]: [Parent 10122, Main Thread] WARNING: failed to create memfd: Too many open files: file /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/base/shared_memory_posix.cc:306 Nov 07 19:56:03 host firefox.desktop[10122]: ExceptionHandler::GenerateDump sys_pipe failed:Too many open files Nov 07 19:56:03 host firefox.desktop[10122]: ExceptionHandler::GenerateDump cloned child 51639 Nov 07 19:56:03 host firefox.desktop[51639]: ExceptionHandler::WaitForContinueSignal waiting for continue signal... Nov 07 19:56:03 host firefox.desktop[10122]: ExceptionHandler::SendContinueSignalToChild sys_write failed: Nov 07 19:56:03 host firefox.desktop[51639]: ExceptionHandler::WaitForContinueSignal sys_read failed: Nov 07 19:56:03 host firefox.desktop[10122]: Bad file descriptor Nov 07 19:56:03 host firefox.desktop[10122]: ExceptionHandler::SendContinueSignalToChild sent continue signal to child Any ideas?
Looks like Firefox leaks file descriptors which means it can't operate after while when fd pool is depleted. Can you try: 1) New/Clean profile without extensions? https://fedoraproject.org/wiki/How_to_debug_Firefox_problems?rd=Bug_info_Firefox#Test_Firefox_with_a_new_profile 2) Latest nightly from Mozilla: https://fedoraproject.org/wiki/How_to_debug_Firefox_problems?rd=Bug_info_Firefox#Testing_Mozilla_binaries I expect this is recent change when switched from FF 93 to 94, right?
Hi Martin, (In reply to Martin Stransky from comment #1) > Can you try: > > 1) New/Clean profile without extensions? > https://fedoraproject.org/wiki/ > How_to_debug_Firefox_problems?rd=Bug_info_Firefox#Test_Firefox_with_a_new_pro > file I'm trying that right now. So far I've only seen the message about mesa_glthread being overridden. > I expect this is recent change when switched from FF 93 to 94, right? Exactly. I'm using a second machine on which the firefox profile stays in sync with the first one and I haven't experienced any crashes so far. Bookmarks, extensions, etc. where copied over a few years ago and I've been using Firefox Sync since it appeared. I do see the messages about MediaDecoderStateMachine on the second one too. Would the list of my extensions be of any use?
Created attachment 1841100 [details] journal contents from the time of the crash The clean profile crashed as well. I saw that abrt had actually uploaded something to the retrace server on multiple occasions in the last six days and it points to these three reports: https://retrace.fedoraproject.org/faf/reports/116384/ https://retrace.fedoraproject.org/faf/reports/224532/ https://retrace.fedoraproject.org/faf/reports/129920/ I think that different problems have been grouped together though, I never had any crashes in the time span covered by these reports, nothing before upgrading to Firefox 94. I am attaching my system log from the moment of the crash and while I was fiddling with the "restore tab" buttons, I hope you can get something out of it.
Question: I've always been using the firefox package, even though my GNOME session is running on X11. Did something change there with the 94 release? Should I be using the firefox-x11 package to explicitly specify the X11 backend?
(In reply to Alexander Ploumistos from comment #4) > Question: I've always been using the firefox package, even though my GNOME > session is running on X11. Did something change there with the 94 release? > Should I be using the firefox-x11 package to explicitly specify the X11 > backend? Nope, that's not it: Nov 11 17:27:02 host firefox-x11.desktop[14201]: [Parent 14201, IPC I/O Parent] WARNING: Message needs unreceived descriptors channel:7fc1ad8e1700 message-type:3604483 header()->num_fds:1 num_fds:0 fds_i:0: file /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/chrome/common/ipc_channel_posix.cc:502 Nov 11 17:27:02 host firefox-x11.desktop[16152]: Exiting due to channel error.
I see. Please run Firefox on terminal with: MOZ_LOG="PlatformDecoderModule:5" env variable set, play any video and attach the log here. Thanks.
Created attachment 1841403 [details] Firefox STDOUT & STDERR Here it is. Would it be useful to leave logging turned on until there's a crash and then upload just the last few lines? In the meantime I've upgraded to F35 and the crashes are still there - I am changing the version.
The log looks ok. Can you reproduce it with latest nightly from Mozilla? https://fedoraproject.org/wiki/How_to_debug_Firefox_problems?rd=Bug_info_Firefox#Testing_Mozilla_binaries
Created attachment 1841427 [details] STDOUT from Firefox Nightly (In reply to Martin Stransky from comment #8) > The log looks ok. Can you reproduce it with latest nightly from Mozilla? I grabbed 96.0a1, built on 2021-11-11 and a few minutes ago it too crashed: Nov 12 12:40:30 host kernel: Web Content[15876]: segfault at 0 ip 00007f53e2aba34e sp 00007ffdc23c1f70 error 6 in libxul.so[7f53ddc66000+5bdb000] Nov 12 12:40:30 host kernel: Code: 8b 0d 1e 43 a4 03 48 89 01 c7 04 25 00 00 00 00 7f 00 00 00 e8 4b 53 1b fb 48 8d 05 1e 23 e3 00 48 8b 0d fd 42 a4 03 48 89 01 <c7> 04 25 00 00 00 00 21 00 00 00 e8 2a 53 1b fb 48 8d 05 f4 11 17 There were another couple of child processes that dumped core with the same message in the system log. The errors in Firefox's output seem identical to the previous ones.
*** Bug 2022506 has been marked as a duplicate of this bug. ***
Looks like Firefox is leaking file descriptors but I have no idea where. There are indications that it's related to media playback but the log looks ok (I'd expect it's related to dmabuf backend). According to https://retrace.fedoraproject.org/faf/reports/116384/ it begins in July which matches Firefox 90 release.
Do you have installed ffmpeg from rpmfusion? It may be related to openh264 and/or bundled ffvpx. It can be also related to RDD process - it's sandboxed process for SW decoders. You can try to disable it, at about:config page set media.rdd-process.enabled to false and restart the browser. But disabling the RDD process sandbox is a security risk (decoders will run in content process).
(In reply to Martin Stransky from comment #11) > Looks like Firefox is leaking file descriptors but I have no idea where. > There are indications that it's related to media playback but the log looks > ok (I'd expect it's related to dmabuf backend). The "error 6 in libxul.so" is a symptom or a cause? > According to > https://retrace.fedoraproject.org/faf/reports/116384/ it begins in July > which matches Firefox 90 release. In my case, I haven't had a single tab crash in maybe two years. I'm regularly checking koji for new Firefox builds and more often than not, I install them before you submit them to bodhi. I'm fairly certain that I didn't have any issues prior to v94. (In reply to Martin Stransky from comment #12) > Do you have installed ffmpeg from rpmfusion? It may be related to openh264 > and/or bundled ffvpx. I do, but wouldn't there be more similar reports if that were the culprit? > It can be also related to RDD process - it's sandboxed process for SW > decoders. You can try to disable it, > at about:config page set media.rdd-process.enabled to false and restart the > browser. > But disabling the RDD process sandbox is a security risk (decoders will run > in content process). Is there any chance that the "Firefox Multi-Account Containers" add-on increases the use of file descriptors? It's the only add-on that was updated around the time I started getting the crashes (Nov 2).
I just had Firefox crash completely. This time there was no video playback, I had left an HTML5 game running in the background, but nothing was happening; I went to log out and noticed that the interface was lagging and then everything crashed. Nov 15 20:10:53 host firefox.desktop[3599]: [Parent 3599, Main Thread] WARNING: failed to create memfd: Too many open files: file /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/base/shared_memory_posix.cc:306 Nov 15 20:10:53 host firefox.desktop[3599]: ExceptionHandler::GenerateDump sys_pipe failed:Too many open files Nov 15 20:10:53 host firefox.desktop[3599]: ExceptionHandler::GenerateDump cloned child 10128 Nov 15 20:10:53 host firefox.desktop[3599]: ExceptionHandler::SendContinueSignalToChild sys_write failed: Nov 15 20:10:53 host firefox.desktop[10128]: ExceptionHandler::WaitForContinueSignal waiting for continue signal... Nov 15 20:10:53 host firefox.desktop[10128]: ExceptionHandler::WaitForContinueSignal sys_read failed: Nov 15 20:10:53 host firefox.desktop[3599]: Bad file descriptor Nov 15 20:10:53 host firefox.desktop[3599]: ExceptionHandler::SendContinueSignalToChild sent continue signal to child Nov 15 20:10:53 host firefox.desktop[10128]: Bad file descriptor Nov 15 20:10:53 host firefox.desktop[4558]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[4521]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[5161]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[3991]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[3926]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[3810]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[5244]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[3814]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[3806]: Exiting due to channel error. Nov 15 20:10:53 host firefox.desktop[3764]: Exiting due to channel error.
(In reply to Alexander Ploumistos from comment #14) > I just had Firefox crash completely. This time there was no video playback, > I had left an HTML5 game running in the background, but nothing was > happening; I went to log out and noticed that the interface was lagging and > then everything crashed. Which game is that? It may be also related to WebGL and dmabuf backend.
(In reply to Alexander Ploumistos from comment #13) > (In reply to Martin Stransky from comment #11) > > Looks like Firefox is leaking file descriptors but I have no idea where. > > There are indications that it's related to media playback but the log looks > > ok (I'd expect it's related to dmabuf backend). > > The "error 6 in libxul.so" is a symptom or a cause? I think it's a symptom. > > According to > > https://retrace.fedoraproject.org/faf/reports/116384/ it begins in July > > which matches Firefox 90 release. > > In my case, I haven't had a single tab crash in maybe two years. I'm > regularly checking koji for new Firefox builds and more often than not, I > install them before you submit them to bodhi. I'm fairly certain that I > didn't have any issues prior to v94. May be also related to EGL which is new in FF94, you can try turn it off: https://fedoraproject.org/wiki/How_to_debug_Firefox_problems?rd=Bug_info_Firefox#Check_Firefox_X11_GLX.2FEGL_backend > (In reply to Martin Stransky from comment #12) > > Do you have installed ffmpeg from rpmfusion? It may be related to openh264 > > and/or bundled ffvpx. > > I do, but wouldn't there be more similar reports if that were the culprit? Yes, that looks unlikely. > > It can be also related to RDD process - it's sandboxed process for SW > > decoders. You can try to disable it, > > at about:config page set media.rdd-process.enabled to false and restart the > > browser. > > But disabling the RDD process sandbox is a security risk (decoders will run > > in content process). > > Is there any chance that the "Firefox Multi-Account Containers" add-on > increases the use of file descriptors? It's the only add-on that was updated > around the time I started getting the crashes (Nov 2). I have little experience with addons so I can't say.
It looks to me the crashing gets reproducible/fixed when installing/deinstalling: fdk-aac ffmpeg-libs opencore-amr vo-amrwbenc x264-libs x265-libs xvidcore chromium-freeworld ffmpeg gstreamer1-libav gstreamer1-plugins-bad-freeworld libavdevice mplayer simplescreenrecorder vlc Such a good reproducer (although not 100%) is for me: https://www.idnes.cz/zpravy/archiv Unfortunately the reproducibility is not quick to easily reduce the number of packages (I somehow cannot confirm it is just due to gstreamer1-plugins-bad-freeworld). Surprisingly this number is always 600-700 no matter if it is crashing or not: for i in `pidof firefox`;do ls -l /proc/$i/fd/;done|wc -l
(In reply to Martin Stransky from comment #16) > May be also related to EGL which is new in FF94, you can try turn it off: > https://fedoraproject.org/wiki/ > How_to_debug_Firefox_problems?rd=Bug_info_Firefox#Check_Firefox_X11_GLX. > 2FEGL_backend That seems to be it. I've opened multiple videos, about 30 tabs with heavy pages and games, not a single error message about file descriptors, no crash since the moment you suggested it. If it helps, I'm running Firefox on the integrated graphics of my laptop, which according to lspci is 00:02.0 VGA compatible controller: Intel Corporation HD Graphics 630 (rev 04) which is using the i915 kernel module. I also have # rpm -qa | grep intel | sort intel-gmmlib-21.3.2-1.fc35.x86_64 intel-media-driver-21.3.4-1.fc35.x86_64 intel-mediasdk-21.3.5-1.fc35.x86_64 libva-intel-driver-2.4.1-6.fc35.x86_64 xorg-x11-drv-intel-2.99.917-51.20200205.fc35.x86_64 Perhaps @tseewald could provide some system info to see if there's any connection. Martin, is there any more information I could provide that would be useful?
Thanks, I'll test EGL on X11 then.
Just out of curiosity, can you try to run Firefox in EGL mode but set widget.dmabuf-webgl.enabled to false at about:config?
(In reply to Martin Stransky from comment #20) > Just out of curiosity, can you try to run Firefox in EGL mode but set > widget.dmabuf-webgl.enabled to false at about:config? So far so good. I've double-checked that I'm indeed running on EGL and I don't see anything about unconsumed/unreceived/invalid file descriptors. I'll keep an eye out for any problems, but it seems your hunch was correct.
Update: Since yesterday, I've had 5 occurrences of firefox.desktop[12009]: [Parent 12009, IPC I/O Parent] WARNING: FileDescriptorSet destroyed with unconsumed descriptors: file /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/chrome/common/file_descriptor_set_posix.cc:19 in my log, but still no crash.
Yes, I can reproduce that with EGL / X11 even on Fedora 34. (In reply to Alexander Ploumistos from comment #22) > Update: > Since yesterday, I've had 5 occurrences of > > firefox.desktop[12009]: [Parent 12009, IPC I/O Parent] WARNING: > FileDescriptorSet destroyed with unconsumed descriptors: file > /builddir/build/BUILD/firefox-94.0/ipc/chromium/src/chrome/common/ > file_descriptor_set_posix.cc:19 > > in my log, but still no crash. This comes from IPC bridge and it's not relevant here. Thanks.
Upstream bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1739924
Guys, can you attach your about:support from box where you can reproduce it?
(In reply to Martin Stransky from comment #25) > Guys, can you attach your about:support from box where you can reproduce it? Text or JSON? I've been running lsof in a loop for the past hour waiting for a crash, in order to provide the info to mozilla's bugtracker.
Created attachment 1842642 [details] about:support info
Thanks, the widget.dmabuf-webgl.enabled test you did was confirmed so it's a bug in dmabuf/sync/intel.
https://bugzilla.mozilla.org/show_bug.cgi?id=1741997 has a fix for it.
firefox-94.0-2.* builds should fix that.
FEDORA-2021-15210f4cb2 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-15210f4cb2
FEDORA-2021-af31f7a256 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-af31f7a256
FEDORA-2021-b033c2d885 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-b033c2d885
FEDORA-2021-af31f7a256 has been pushed to the Fedora 34 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-af31f7a256` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-af31f7a256 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-15210f4cb2 has been pushed to the Fedora 35 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-15210f4cb2` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-15210f4cb2 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-b033c2d885 has been pushed to the Fedora 33 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-b033c2d885` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-b033c2d885 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2021-15210f4cb2 has been pushed to the Fedora 35 stable repository. If problem still persists, please make note of it in this bug report.
(In reply to Jan Kratochvil from comment #17) > It looks to me the crashing gets reproducible/fixed when > installing/deinstalling: ... ffmpeg-libs was the culprit but that looks as a different Bug, filed as: Bug 2025902
FEDORA-2021-af31f7a256 has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2022-f0988ea008 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2022-f0988ea008
FEDORA-2022-f0988ea008 has been pushed to the Fedora 38 stable repository. If problem still persists, please make note of it in this bug report.