Bug 1889251

Summary: Firefox 81 crashes in mozilla::widget::WaylandShmPool::WaylandShmPool
Product: [Fedora] Fedora Reporter: Tom Hughes <tom>
Component: firefoxAssignee: Martin Stransky <stransky>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 32CC: elxreno, erack, gecko-bugs-nobody, jhorak, john.j5live, kai-engert-fedora, petersen, pjasicek, rhughes, rmeggins, rstrode, sandmann, stransky, tqueiros
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: firefox-82.0.2-1.fc32 firefox-82.0.2-1.fc31 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-31 02:01:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
coredumpctl report none

Description Tom Hughes 2020-10-19 07:29:36 UTC
Created attachment 1722570 [details]
coredumpctl report

Description of problem:

Since firefox updated to version 81 I am seeing it crashing multiple times a day. I will attach the full report from coredumpctl (gathered with the mozilla crash reporter disabled) but the crash appears to be::

  #0  0x00007f422c699915 raise (libpthread.so.0 + 0x14915)
  #1  0x00007f4224e22f7f nsProfileLock::FatalSignalHandler(int, siginfo_t*, void*) (libxul.so + 0x26aff7f)
  #2  0x00007f422c699a90 __restore_rt (libpthread.so.0 + 0x14a90)
  #3  0x00007f42244d76d5 mozilla::widget::WaylandShmPool::WaylandShmPool(mozilla::widget::nsWaylandDisplay*, int) (libxul.so + 0x1d646d5)
  #4  0x00007f42244d7b1a mozilla::widget::WindowBackBufferShm::WindowBackBufferShm(mozilla::widget::WindowSurfaceWayland*, int, int) (libxul.so + 0x1d64b1a)
  #5  0x00007f42244d7d85 mozilla::widget::WindowSurfaceWayland::CreateWaylandBuffer(int, int) (libxul.so + 0x1d64d85)
  #6  0x00007f42244d7e8f mozilla::widget::WindowSurfaceWayland::SetNewWaylandBuffer() (libxul.so + 0x1d64e8f)
  #7  0x00007f42244dd651 mozilla::widget::WindowSurfaceWayland::LockWaylandBuffer() (libxul.so + 0x1d6a651)
  #8  0x00007f42244deb6a mozilla::widget::WindowSurfaceWayland::Lock(mozilla::gfx::IntRegionTyped<mozilla::LayoutDevicePixel> const&) (libxul.so + 0x1d6bb6a)
  #9  0x00007f4225d316eb mozilla::widget::WindowSurfaceProvider::StartRemoteDrawingInRegion(mozilla::gfx::IntRegionTyped<mozilla::LayoutDevicePixel>&, mozilla::layers::BufferMode*) (libxul.so + 0x35be6eb)
  #10 0x00007f4225780861 mozilla::layers::BasicCompositor::BeginFrameForWindow(mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::Maybe<mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> > const&, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&) (libxul.so + 0x300d861)
  #11 0x00007f42257e5a6e mozilla::layers::LayerManagerComposite::Render(mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::IntRegionTyped<mozilla::gfx::UnknownUnits> const&) (libxul.so + 0x3072a6e)
  #12 0x00007f42257e55b6 mozilla::layers::LayerManagerComposite::UpdateAndRender() (libxul.so + 0x30725b6)
  #13 0x00007f42257e5387 mozilla::layers::LayerManagerComposite::EndTransaction(mozilla::TimeStamp const&, mozilla::layers::LayerManager::EndTransactionFlags) (libxul.so + 0x3072387)
  #14 0x00007f42257f1452 mozilla::layers::CompositorBridgeParent::CompositeToTarget(mozilla::layers::BaseTransactionId<mozilla::VsyncIdType>, mozilla::gfx::DrawTarget*, mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits> const*) (libxul.so + 0x307e452)
  #15 0x00007f42257f110c mozilla::layers::CompositorVsyncScheduler::Composite(mozilla::VsyncEvent const&) (libxul.so + 0x307e10c)
  #16 0x00007f42257f1014 mozilla::detail::RunnableMethodImpl<mozilla::layers::CompositorVsyncScheduler*, void (mozilla::layers::CompositorVsyncScheduler::*)(mozilla::VsyncEvent const&), true, (mozilla::RunnableKind)1, mozilla::VsyncEvent>::Run() (libxul.so + 0x307e014)
  #17 0x00007f4225135fd1 nsThread::ProcessNextEvent(bool, bool*) (libxul.so + 0x29c2fd1)
  #18 0x00007f4225135d60 NS_ProcessNextEvent(nsIThread*, bool) (libxul.so + 0x29c2d60)
  #19 0x00007f422514d648 mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) (libxul.so + 0x29da648)
  #20 0x00007f4225565515 MessageLoop::Run() (libxul.so + 0x2df2515)
  #21 0x00007f42253f6897 nsThread::ThreadFunc(void*) (libxul.so + 0x2c83897)
  #22 0x00007f422b6e7554 _pt_root (libnspr4.so + 0x2c554)
  #23 0x00007f422c68e432 start_thread (libpthread.so.0 + 0x9432)
  #24 0x00007f422c264913 __clone (libc.so.6 + 0x101913)

I think this is the upstream data from the mozilla crash reporter:

https://crash-stats.mozilla.org/signature/?signature=mozilla%3A%3Awidget%3A%3AWaylandShmPool%3A%3AWaylandShmPool&date=%3E%3D2020-09-18T09%3A16%3A00.000Z&date=%3C2020-10-18T09%3A16%3A00.000Z&_columns=date&_columns=product&_columns=version&_columns=build_id&_columns=platform&_columns=reason&_columns=address&_columns=install_time&_columns=startup_crash&_sort=-date&page=1#summary

That appears to agree that it's new in 81 and mostly affecting Fedora though that may just be due to it being Wayland related.

Version-Release number of selected component (if applicable):

firefox-81.0.2-1.fc32.x86_64 (but seem with all 81.x versions to date)

How reproducible:

Happens probably 2-3 times a day.

Comment 1 Rich Megginson 2020-10-19 20:30:57 UTC
I have the same version of firefox/fedora.  I don't see any crashes reported by coredumpctl, but my firefox "crashes" several times a day - I don't see a core, but firefox exits, and I see messages like this in the journal:

Oct 19 14:09:56 localhost.localdomain firefox.desktop[167195]: ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost

Comment 2 Tom Hughes 2020-10-19 21:27:27 UTC
You won't see the crash in coredumpctl unless you run with MOZ_CRASHREPORTER_DISABLE=1 set in the environment as firefox has it's own builtin crash reporter that kicks in instead.

Comment 3 Rich Megginson 2020-10-20 01:59:09 UTC
I didn't see a core dump, but I did see this message:

IPDL protocol error: Handler returned error code!

###!!! [Parent][DispatchAsyncMessage] Error: PLayerTransaction::Msg_ReleaseLayer Processing error: message was deserialized, but the handler returned false (indicating failure)

Gdk-Message: 19:54:18.997: Error flushing display: Broken pipe
Exiting due to channel error.
Crash Annotation GraphicsCriticalError: |[C0][GFX1-]: Receive IPC close with reason=AbnormalShutdown (t=8183.13) Exiting due to channel error.
Exiting due to channel error.

so I'm not sure if this is https://bugzilla.redhat.com/show_bug.cgi?id=1888920

Comment 4 Rich Megginson 2020-10-24 15:40:56 UTC
In my case I don't have multiple workspaces, but it seems to be related to switching windows e.g. using Alt-Tab or using the hotspot with the mouse.

It's still crashing - I just updated to firefox 82.0-4 on f32

This is getting to be a problem - crashing several times a day - I'm considering switching to chrome (and as a former Netscape employee, that kills me to write)

Comment 5 Tom Hughes 2020-10-24 15:55:00 UTC
I also saw my crashes persist after updating to Firefox 82 but the good news is that yesterday I updated to Fedora 33 and 30 hours later I haven't had a single crash yet.

It might be that a reboot, or at least a desktop restart, would have the same effect in Fedora 32 if it's related to the wider desktop stack. I agree that things like switching desktop appeared to be a trigger for me, as did things like trying to save a download, but sometime they just seemed to be spontaneous.

Comment 6 Martin Stransky 2020-10-25 21:05:50 UTC
(In reply to Rich Megginson from comment #4)
> In my case I don't have multiple workspaces, but it seems to be related to
> switching windows e.g. using Alt-Tab or using the hotspot with the mouse.
> 
> It's still crashing - I just updated to firefox 82.0-4 on f32
> 
> This is getting to be a problem - crashing several times a day - I'm
> considering switching to chrome (and as a former Netscape employee, that
> kills me to write)

See https://fedoramagazine.org/firefox-tips-for-fedora-31/
You can switch back to X11 backend for Firefox if Wayland is too raw for you.

Comment 7 Martin Stransky 2020-10-25 21:32:35 UTC
It seems to happen in WaylandAllocateShmMemory() when the Shm allocation fails. Let's make it more flexible in such case and don't crash.

Comment 8 Tom Hughes 2020-10-25 21:35:24 UTC
I guessed it might be a shm allocation failure given the backtrace, but as far as I could see I was nowhere near any limits on shm availability. I have 32Gb of RAM and a 16Gb limit on shm and firefox was only using a few hundred megabytes.

Comment 9 Martin Stransky 2020-10-25 22:12:34 UTC
(In reply to Tom Hughes from comment #8)
> I guessed it might be a shm allocation failure given the backtrace, but as
> far as I could see I was nowhere near any limits on shm availability. I have
> 32Gb of RAM and a 16Gb limit on shm and firefox was only using a few hundred
> megabytes.

Good to know. OTOH Firefox may have a sandbox restriction on Shm usage.

Comment 10 Martin Stransky 2020-10-26 14:26:40 UTC
Upstream bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1673313

Comment 11 Tom Hughes 2020-10-26 14:30:08 UTC
As it happens this morning I had a crash (some 72 hours after rebooting) with Fedora 33/Firefox 82. I didn't have the crash reporter disabled so I can't say for sure it's the same crash.

Comment 12 Martin Stransky 2020-10-27 13:38:11 UTC
The fix is backported to firefox-82.0-8 builds.

Comment 13 Fedora Update System 2020-10-29 07:58:06 UTC
FEDORA-2020-bcfc7810a7 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-bcfc7810a7

Comment 14 Fedora Update System 2020-10-29 07:58:07 UTC
FEDORA-2020-9365bd146c has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-9365bd146c

Comment 15 Fedora Update System 2020-10-30 01:30:39 UTC
FEDORA-2020-9365bd146c has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-9365bd146c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9365bd146c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Fedora Update System 2020-10-30 02:02:06 UTC
FEDORA-2020-bcfc7810a7 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-bcfc7810a7`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-bcfc7810a7

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 17 Jens Petersen 2020-10-30 02:53:15 UTC
Thanks, that is good news, though I haven't seen any crashes since tweaking my earlyoom settings.

Comment 18 Fedora Update System 2020-10-31 02:01:53 UTC
FEDORA-2020-4a0e4504c6 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 19 Fedora Update System 2020-10-31 03:16:12 UTC
FEDORA-2020-871455fdcf has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-871455fdcf`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-871455fdcf

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 20 Fedora Update System 2020-11-04 03:54:47 UTC
FEDORA-2020-871455fdcf has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.