Created attachment 2127731 [details]
system log with gstreamer1-1.26.10-2.fc44

Created attachment 2127732 [details]
system log with gstreamer1-1.28.0-1.fc44

Proposed as a Freeze Exception for 44-beta by Fedora user mikhail using the blocker tracking app because:

 Regression: after updating gstreamer1 to 1.28.0-1.fc44, GNOME’s default screenshot UI loses the “Record Screen” option (only “Take Screenshot” remains). This removes an important end-user workflow (screencasts for bug reports / documentation). Workaround requires downgrading gstreamer1 and re-logging, so this should be tracked as a Freeze Exception for Fedora 44 Beta.

After the previous findings around gstreamer1, the issue now reproduces again after updating PipeWire in Fedora Rawhide/F44.

Today I received pipewire-1.5.85-3.fc44. Updating PipeWire to 1.5.85-3.fc44 also makes GNOME’s built-in screenshot/screencast UI lose the “Record Screen” option (only “Take Screenshot” remains). At this point, to restore the feature I have to downgrade both GStreamer (as previously described) and PipeWire.

Workaround that restores “Record Screen” (and screenshot UI functionality) is downgrading PipeWire back to 1.5.85-2.fc44:
# sudo dnf downgrade \
  pipewire-0:1.5.85-2.fc44 \
  pipewire-alsa-0:1.5.85-2.fc44 \
  pipewire-config-raop-0:1.5.85-2.fc44 \
  pipewire-gstreamer-0:1.5.85-2.fc44 \
  pipewire-jack-audio-connection-kit-0:1.5.85-2.fc44 \
  pipewire-jack-audio-connection-kit-libs-0:1.5.85-2.fc44 \
  pipewire-libs-0:1.5.85-2.fc44 \
  pipewire-plugin-libcamera-0:1.5.85-2.fc44 \
  pipewire-pulseaudio-0:1.5.85-2.fc44 \
  pipewire-utils-0:1.5.85-2.fc44 \
  libcamera-0:0.6.0-2.fc44 \
  libcamera-ipa-0:0.6.0-2.fc44 --refresh


I did not reliably observe the following error during this PipeWire regression (or I may have missed it), but mentioning it for reference:
org.gnome.Shell.Screencast[5642]: JS ERROR: Error: Expected type utf8 for Argument 'argv' but got type 'null'

This suggests the regression might involve the PipeWire / screencast integration path (possibly via pipewire-gstreamer or related components), not only GStreamer core.

So this is broken with any or both of gstreamer1 or pipewire upgraded? Or just pipewire?

(In reply to Gwyn Ciesla from comment #5)
> So this is broken with any or both of gstreamer1 or pipewire upgraded? Or
> just pipewire?

# rpm -q gstreamer1 pipewire       |
gstreamer1-1.26.10-2.fc44.x86_64   |
gstreamer1-1.26.10-2.fc44.i686     | “Record Screen” - available
pipewire-1.5.85-2.fc44.x86_64      |
pipewire-1.5.85-2.fc44.i686        |
------------------------------------------------------------------
# rpm -q gstreamer1 pipewire       |
gstreamer1-1.26.10-2.fc44.x86_64   |
gstreamer1-1.26.10-2.fc44.i686     | “Record Screen” - unavailable
pipewire-1.5.85-3.fc44.x86_64      |
pipewire-1.5.85-3.fc44.i686        |
------------------------------------------------------------------
# rpm -q gstreamer1 pipewire       |
gstreamer1-1.28.0-1.fc44.x86_64    |
gstreamer1-1.28.0-1.fc44.i686      | “Record Screen” - unavailable
pipewire-1.5.85-3.fc44.x86_64      |
pipewire-1.5.85-3.fc44.i686        |
------------------------------------------------------------------
# rpm -q gstreamer1 pipewire       |
gstreamer1-1.26.10-2.fc44.x86_64   |
gstreamer1-1.26.10-2.fc44.i686     | “Record Screen” - unavailable
pipewire-1.5.85-3.fc44.x86_64      |
pipewire-1.5.85-3.fc44.i686        |

Could we consider adding an openQA check that verifies GNOME’s screenshot overlay still provides “Record Screen” and that a short screencast can be started/stopped successfully (e.g. produces a .webm in the default Videos/Screencasts location)?
This regression is particularly painful for QA because working screen recording is essential for bug reporting and debugging; catching it in openQA would prevent this class of breakage from slipping through again.

We can consider it, sure. File a ticket at https://forge.fedoraproject.org/quality/os-autoinst-distri-fedora/ and it'll be in the Big Backlog Of Good Test Ideas...

What about?

gstreamer1-1.28.0-1.fc44.x86_64    
gstreamer1-1.28.0-1.fc44.i686      
pipewire-1.5.85-2.fc44.x86_64      
pipewire-1.5.85-2.fc44.i686

(In reply to Gwyn Ciesla from comment #9)
> What about?
> 
> gstreamer1-1.28.0-1.fc44.x86_64    
> gstreamer1-1.28.0-1.fc44.i686      
> pipewire-1.5.85-2.fc44.x86_64      
> pipewire-1.5.85-2.fc44.i686

# rpm -q gstreamer1 pipewire       |
gstreamer1-1.28.0-1.fc44.x86_64    |
gstreamer1-1.28.0-1.fc44.i686      | “Record Screen” - unavailable
pipewire-1.5.85-2.fc44.x86_64      |
pipewire-1.5.85-2.fc44.i686        |

After updating gnome-shell and mutter, the “Record Screen” option is available again in the GNOME screenshot overlay.

Notably, this happened without downgrading GStreamer — gstreamer1 is still at 1.28.0-1.fc44.

Current package versions:
# rpm -qa | grep "gstreamer1\|mutter\|gnome-shell" | sort
gnome-shell-50~beta-3.fc45.x86_64
gnome-shell-common-50~beta-3.fc45.noarch
gnome-shell-extension-appindicator-61-2.fc44.noarch
gnome-shell-extension-apps-menu-50~alpha-1.fc45.noarch
gnome-shell-extension-background-logo-50~beta-2.fc45.noarch
gnome-shell-extension-common-50~alpha-1.fc45.noarch
gnome-shell-extension-launch-new-instance-50~alpha-1.fc45.noarch
gnome-shell-extension-places-menu-50~alpha-1.fc45.noarch
gnome-shell-extension-window-list-50~alpha-1.fc45.noarch
gstreamer1-1.28.0-1.fc44.i686
gstreamer1-1.28.0-1.fc44.x86_64
gstreamer1-plugin-dav1d-0.14.0-3.fc45.x86_64
gstreamer1-plugin-gtk4-0.14.4-2.fc44.x86_64
gstreamer1-plugin-libav-1.28.0-1.fc44.x86_64
gstreamer1-plugin-openh264-1.28.0-4.fc45.x86_64
gstreamer1-plugins-bad-free-1.28.0-4.fc45.x86_64
gstreamer1-plugins-bad-free-libs-1.28.0-4.fc45.x86_64
gstreamer1-plugins-base-1.28.0-1.fc44.x86_64
gstreamer1-plugins-good-1.28.0-1.fc44.x86_64
gstreamer1-plugins-good-gtk-1.28.0-1.fc44.x86_64
gstreamer1-plugins-good-qt-1.28.0-1.fc44.x86_64
gstreamer1-plugins-good-qt6-1.28.0-1.fc44.x86_64
gstreamer1-plugins-ugly-free-1.28.0-1.fc44.x86_64
mutter-50~beta-1.fc45.x86_64
mutter-common-50~beta-1.fc45.noarch

This suggests the missing “Record Screen” option may have been a compatibility/regression interaction that required GNOME Shell/Mutter changes (now present in GNOME 50). If so, it would be great to avoid pushing gstreamer1/pipewire updates without the matching GNOME Shell/Mutter update (or otherwise ensure these interdependent updates land together), to prevent users from hitting broken intermediate states.

Do we need to treat these as a coordinated update set (e.g. side-tag / grouped update / gating) for the branched release?

I was under the impression that they were independent projects?

This seems to be working well here:

gnome-shell-50~beta-4.fc44.x86_64
gstreamer1-1.28.0-1.fc44.x86_64
pipewire-1.5.85-3.fc44.x86_64

Can we close this?

Closing, reopen if still broken, thanks.

One thing I’d like to note for the future: during the regression window it looked like gstreamer1 / pipewire updates landed earlier, while the GNOME Shell/Mutter update (that restores “Record Screen”) was not yet available in the main repo. That created a broken intermediate state for users who updated regularly.

If updating gstreamer1 / pipewire without the matching GNOME Shell/Mutter update can cause this kind of regression, would it make sense to coordinate these updates as a single update set (side-tag / grouped update / gating), so they don’t get pushed independently? If a hard RPM-level dependency is not appropriate, even a packaging or gating-level coordination would help avoid users hitting incompatible combinations.

For reference, the working combo reported here is:
- gnome-shell-50~beta-4.fc44.x86_64
- gstreamer1-1.28.0-1.fc44.x86_64
- pipewire-1.5.85-3.fc44.x86_64

Thanks again.

Hard RPM-level dependencies are possible, but probably overkill. Simply coordinating the updates together would have avoided this. A side tag is one good way to do this, but not even required! As long as the builds are all attached to the same bodhi update, everything should be fine. But of course, this only works if the packager knows to expect the incompatibility....

Thanks, that makes sense.

I agree hard RPM-level dependencies would likely be overkill here. Coordinating via a single Bodhi update (or a side tag) seems like the right approach.

To help ensure we don’t hit the same broken intermediate state again, could we please document this incompatibility somewhere packagers will notice (even just a short note in the update/commit message / packager notes), i.e. that pushing gstreamer1/pipewire updates without the matching GNOME Shell/Mutter build can temporarily remove “Record Screen”? That way, next time these components change again, maintainers will know to ship them as a coordinated update set.

Thanks again for the quick turnaround.

A side tag is actually the only way to do multi-package updates for Rawhide / pre-Bodhi-enablement Branched, so in a sense it would have been "required" in this case.

(In reply to Mikhail from comment #17)
> To help ensure we don’t hit the same broken intermediate state again, could
> we please document this incompatibility somewhere packagers will notice
> (even just a short note in the update/commit message / packager notes), i.e.
> that pushing gstreamer1/pipewire updates without the matching GNOME
> Shell/Mutter build can temporarily remove “Record Screen”? That way, next
> time these components change again, maintainers will know to ship them as a
> coordinated update set.

That's definitely not supposed to happen, and I don't understand why it happened in this case. Nowhere in this bug report did we debug and identify what went wrong, so if it's likely to ever happen again, we don't know. Some feedback from the Pipewire developers would be helpful.

(In reply to Michael Catanzaro from comment #16)
> Hard RPM-level dependencies are possible, but probably overkill.

Why are those overkill? It would prevent these situations, and also we wouldn't need to rely on some instructions documented somewhere that people are likely to overlook anyway.

Furthermore, from my experience, debugging older versions of gnome-shell is a terrible experience (at least in my experience) exactly due to undeclared dependencies like these (and stuff like mutter). If I need to compare functionality with an older version, I often give up in advance, because lining up everything just by trial and error is extremely time-consuming. I would very appreciate if we could use RPM deps to make this easier and more reliable.

> Some feedback from the Pipewire developers would be helpful.

I have no idea what went wrong, the projects are sufficiently independent to not cause these issues or so we think.

> A side tag is actually the only way to do multi-package updates for Rawhide / pre-Bodhi-enablement Branched, so in a sense it would have been "required" in this case.

Normally not, you let projects update independently. You only bundle updates when you know they need synchronized updates or when you
need to build them together with new API etc. In this case, there was no need (or so we thought, I don't know what the problem was/is).

Wim, we have journalctl -b logs attached (good vs bad). Are they useful at all? If not, what exact set of logs/debug env vars do you want next time?

> Normally not, you let projects update independently.

I was replying to Michael's "Simply coordinating the updates together would have avoided this. A side tag is one good way to do this, but not even required!" For Rawhide and pre-Bodhi-enablement Branched, a side tag *is* required to achieve "the builds are all attached to the same bodhi update".