Starting with Fedora-28-20180417.n.0 (this passed with Fedora-28-20180416.n.0), after installing FAW from the DVD installer image, boot of the installed system fails: instead of g-i-s or GDM showing up, the "Oh no!" error screen shows up -
Will attach the /var/log tarball here. Looking at the logs, this error is the immediate cause of the failure:
Apr 17 14:22:42 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gnome-shell: Failed to create backend: Failed to initialize renderer: Missing extension for GBM renderer: EGL_KHR_platform_gbm
Apr 17 14:22:42 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gnome-session: gnome-session-binary: WARNING: App 'org.gnome.Shell.desktop' exited with code 1
Apr 17 14:22:42 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gnome-session-binary: Unrecoverable failure in required component org.gnome.Shell.desktop
Apr 17 14:22:42 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gnome-session-binary: WARNING: App 'org.gnome.Shell.desktop' exited with code 1
Apr 17 14:22:42 ibm-p8-kvm-03-guest-02.virt.pnr.lab.eng.rdu2.redhat.com gdm-launch-environment]: pam_unix(gdm-launch-environment:session): session closed for user gdm
I'm not immediately sure what's causing that, though. Assigning to gnome-shell to start with, but this could well turn out to be mutter or mesa or something. Will CC FAW-interested folks.
Proposing for a freeze exception, as obviously we want the day 0 FAW image to work correctly.
Created attachment 1423275 [details]
tarball containing whole of /var/log after test fails
URL of an image to test with, for convenience: https://kojipkgs.fedoraproject.org/compose/branched/Fedora-28-20180417.n.0/compose/AtomicWorkstation/x86_64/iso/Fedora-AtomicWorkstation-ostree-x86_64-28-20180417.n.0.iso
note openQA tests in a qemu VM with 'std' graphics (same as 'vga', I think). Results may differ on bare metal or with a different virtual video device, I guess.
<mclasen> adamw: did mesa-libEGL go missing again ?
<mclasen> there was some glitch where soft deps were interpreted differently in tree compose, or something
<mclasen> maybe that libsolv change that colin was theorizing about has hit f28 ?
<adamw> who knows!
<adamw> we can apply that comps change to f28 easily enough, though, if that should fix it...
It does look like mesa-libEGL went missing:
$ rpm-ostree db diff 5344360af3807647a728366dc59ef23aa5100b4b83013316e0f1c9c9d0031103 d3461fff6aa51d6167ce57919368688cc2512f080048a3e0784e4161a208c701
ostree diff commit old: 5344360af3807647a728366dc59ef23aa5100b4b83013316e0f1c9c9d0031103
ostree diff commit new: d3461fff6aa51d6167ce57919368688cc2512f080048a3e0784e4161a208c701
Should mutter depend on mesa-libEGL and mesa-libGL?
ajax just did a libglvnd build that adds back the mesa-libGL and mesa-libEGL hard requires.
libglvnd-1.0.1-0.5.20180327git5baa1e5.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-d48d3c17a2
I just opened two PRs (one for rawhide and one for f28) to sync the package list for FAW with comps. Note the rawhide one adds in mesa-libEGL, but the f28 one doesn't. So the comps differ slightly for rawhide vs f28, which might be why we are seeing an issue?
Dusty: that's exactly what the discussion in comment #3 was about, I think. Note that PR mclasen linked to changed only f29, not f28.
(In reply to Adam Williamson from comment #9)
> Dusty: that's exactly what the discussion in comment #3 was about, I think.
> Note that PR mclasen linked to changed only f29, not f28.
ok. if someone does decide to create that PR for f28 let me know and I can update comps again.
So, I managed to join the dots on exactly *why* the change from Requires to Recommends caused mesa-libEGL to go missing from the ostree, I think. Basically, because lorax doesn't pull Recommends into install trees, and the FAW ostree is built using the Workstation install tree for its base repo. Full details here: https://bugzilla.redhat.com/show_bug.cgi?id=1569242
So we kinda have three proposed fixes for this now: the libglvnd update, the clutter update in https://bugzilla.redhat.com/show_bug.cgi?id=1568881 , and the idea of making F28's comps specify mesa-libEGL explicitly like Rawhide's.
We probably don't really need all three of those. What do we think is the most actually-correct combination of the three to do? And should we synchronize whatever we decide on with Rawhide?
libglvnd-1.0.1-0.5.20180327git5baa1e5.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-d48d3c17a2
(In reply to Adam Williamson from comment #12)
> So we kinda have three proposed fixes for this now: the libglvnd update, the
> clutter update in https://bugzilla.redhat.com/show_bug.cgi?id=1568881 , and
> the idea of making F28's comps specify mesa-libEGL explicitly like Rawhide's.
> We probably don't really need all three of those. What do we think is the
> most actually-correct combination of the three to do? And should we
> synchronize whatever we decide on with Rawhide?
I'd prefer going down the hard dep route for F28 GA. We did this for F27 and it's just reverting to a known good state. If we do the comps fix, adding mesa-libEGL to Workstation comps, it probably fixes Workstation well enough, but who knows if it fixes all other spins and distro upgrades. I'd sleep better if we have a hard dep for F28 :)
For rawhide and short term, I'd say let's keep the exact same thing as we do for F28, whichever it ends up to be -- if we go for hard dep, then revert the comps change that already went in.
For longer term for rawhide, I'd like to go back to libglvnd having Recommends on mesa-libEGL and figure out the install tree issues so that recommended deps are pulled in. I don't think this is something we should play with during F28 freeze though.
There are still three permutations of "the hard dep route", though: change libglvnd, change clutter, or change both. Which of those do you prefer?
I think both are needed. libglvnd update pulls in mesa-libGL, clutter update pulls in mesa-dri-drivers.
OK, in that case, I'm +1 FE for both this and 1568881 . Thanks.
That's +4, setting Accepted.
libglvnd-1.0.1-0.5.20180327git5baa1e5.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.
On https://bugzilla.redhat.com/show_bug.cgi?id=1568881#c23 , Petr Pisar wrote:
"After adding mesa-libGL dependency to libglvnd, gtk clients running against Xvfb crash. Probably because of bug #1568644."
As this bug was actually the bug we treated as being 'for libglvnd', let's move that discussion over here. I'm not yet sure how big of a problem this is for F28 release, trying to look into it ATM.
If I'm understanding ajax correctly, he thinks #1568644 is only really a problem if mesa-dri-drivers isn't installed. AFAICT it is installed in all the scenarios that are a problem for F28 release: it's in the installer, it's in desktop live images and so on. (In comps, it's in base-x, and all the desktop environment groups include base-x). I tested a VNC install and it worked OK. So unless we're missing something, we don't think there's a release-critical issue there. The issue with buildroots is a real one, but doesn't need to be a release blocker or FE.