Hide Forgot
Description of problem: Running gnome-shell in vnc session on ppc64le results in black screen. If I run other WM like metacity the output is fine. Version-Release number of selected component (if applicable): gnome-shell-3.22.3-8.el7.ppc64 mutter-3.22.3-4.el7.ppc64 How reproducible: 100% Steps to Reproduce: 1.vncserver 2. connect to vnc session 3. Actual results: Nothing but black screen. Expected results: Session is displayed properly. Additional info:
Created attachment 1268721 [details] journal log journal log for gnome-shell
Created attachment 1268722 [details] syslog
probably a mesa problem? does glxgears work ? can you attach your xorg.conf ?
Not ppc64le specific I can reproduce on ppc64 too (see https://imagebin.ca/v/3HwJdajL8rcA ), it is caused by random VNC session we are using for all our automation for secondary arches I can confirm what Jirka observed: Apr 04 16:03:45 ibm-p740-01-lp1.rhts.eng.bos.redhat.com gnome-shell[9146]: JS ERROR: Exception in callback for signal: desktop-changed: TypeError: this._applicationsButtons.forEach is not a function ApplicationsButton<._init/<@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:430 _emit@resource:///org/gnome/gjs/modules/signals.js:124 DesktopTarget<._setDesktop@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:324 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 DesktopTarget<._onWindowAdded@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:312 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 Btw glxgears is running, I mean no crash and it reports fps. I can even see wheel spinning when gnome-shell is not running.
Extra js output appeared after a while: Apr 04 16:08:41 ibm-p740-01-lp1.rhts.eng.bos.redhat.com gnome-shell[9146]: JS ERROR: Exception in callback for signal: desktop-changed: TypeError: this._applicationsButtons.forEach is not a function ApplicationsButton<._init/<@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:430 _emit@resource:///org/gnome/gjs/modules/signals.js:124 DesktopTarget<._setDesktop@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:324 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 DesktopTarget<.destroy@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:345 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 ApplicationsButton<._onDestroy@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:480 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 Button<.destroy@resource:///org/gnome/shell/ui/panelMenu.js:186 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 _parent@resource:///org/gnome/gjs/modules/lang.js:131 ApplicationsButton<.destroy@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:740 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 disable@/usr/share/gnome-shell/extensions/apps-menu@gnome-shell-extensions.gcampax.github.com/extension.js:756 disableExtension@resource:///org/gnome/shell/ui/extensionSystem.js:82 disableAllExtensions/<@resource:///org/gnome/shell/ui/extensionSystem.js:357 disableAllExtensions@resource:///org/gnome/shell/ui/extensionSystem.js:356 _sessionUpdated@resource:///org/gnome/shell/ui/extensionSystem.js:374 _emit@resource:///org/gnome/gjs/modules/signals.js:124 SessionMode<._sync@resource:///org/gnome/shell/ui/sessionMode.js:205 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 SessionMode<.pushMode@resource:///org/gnome/shell/ui/sessionMode.js:167 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 ScreenShield<.activate@resource:///org/gnome/shell/ui/screenShield.js:1280 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 ScreenShield<._onLongLightboxShown@resource:///org/gnome/shell/ui/screenShield.js:898 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 _emit@resource:///org/gnome/gjs/modules/signals.js:124 Lightbox<.show/<.onComplete<@resource:///org/gnome/shell/ui/lightbox.js:186 _addHandler/params[name]@resource:///org/gnome/shell/ui/tweener.js:91 _callOnFunction@resource:///org/gnome/gjs/modules/tweener/tweener.js:203 _updateTweenByIndex@resource:///org/gnome/gjs/modules/tweener/tweener.js:333 _updateTweens@resource:///org/gnome/gjs/modules/tweener/tweener.js:345 _onEnterFrame@resource:///org/gnome/gjs/modules/tweener/tweener.js:360 _emit@resource:///org/gnome/gjs/modules/signals.js:124 ClutterFrameTicker<._onNewFrame@resource:///org/gnome/shell/ui/tweener.js:208 wrapper@resource:///org/gnome/gjs/modules/lang.js:178 ClutterFrameTicker<._init/<@resource:///org/gnome/shell/ui/tweener.js:183
Created attachment 1268742 [details] xorg.conf
Created attachment 1268743 [details] 10-dummylibvnc.conf
just FYI metacity works
New (rhel7.4 one) mesa: rpm -qa mesa\* mesa-libGL-17.0.1-2.20170307.el7.ppc64 mesa-libOSMesa-17.0.1-2.20170307.el7.ppc64 mesa-demos-8.2.0-3.el7.ppc64 mesa-private-llvm-debuginfo-3.9.1-2.el7.ppc64 mesa-libglapi-17.0.1-2.20170307.el7.ppc64 mesa-libgbm-17.0.1-2.20170307.el7.ppc64 mesa-libGLw-8.0.0-4.el7.ppc64 mesa-libGLw-devel-8.0.0-4.el7.ppc64 mesa-libEGL-devel-17.0.1-2.20170307.el7.ppc64 mesa-private-llvm-devel-3.9.1-2.el7.ppc64 mesa-demos-debuginfo-8.2.0-3.el7.ppc64 mesa-libEGL-17.0.1-2.20170307.el7.ppc64 mesa-filesystem-17.0.1-2.20170307.el7.ppc64 mesa-libGL-devel-17.0.1-2.20170307.el7.ppc64 mesa-libGLU-devel-9.0.0-4.el7.ppc64 mesa-libGLES-devel-17.0.1-2.20170307.el7.ppc64 mesa-libGLw-debuginfo-8.0.0-4.el7.ppc64 mesa-private-llvm-3.9.1-2.el7.ppc64 mesa-libOSMesa-devel-17.0.1-2.20170307.el7.ppc64 mesa-libgbm-devel-17.0.1-2.20170307.el7.ppc64 mesa-debuginfo-17.0.1-2.20170307.el7.ppc64 mesa-libGLU-9.0.0-4.el7.ppc64 mesa-libGLES-17.0.1-2.20170307.el7.ppc64 mesa-dri-drivers-17.0.1-2.20170307.el7.ppc64 mesa-libGLU-debuginfo-9.0.0-4.el7.ppc64 Old mesa: # rpm -qa mesa\* mesa-demos-8.2.0-3.el7.ppc64 mesa-private-llvm-debuginfo-3.9.1-2.el7.ppc64 mesa-libGL-11.2.2-2.20160614.el7.ppc64 mesa-libGLES-devel-11.2.2-2.20160614.el7.ppc64 mesa-libGLw-8.0.0-4.el7.ppc64 mesa-libGLw-devel-8.0.0-4.el7.ppc64 mesa-demos-debuginfo-8.2.0-3.el7.ppc64 mesa-libEGL-11.2.2-2.20160614.el7.ppc64 mesa-filesystem-11.2.2-2.20160614.el7.ppc64 mesa-libOSMesa-devel-11.2.2-2.20160614.el7.ppc64 mesa-private-llvm-devel-3.8.1-1.el7.ppc64 mesa-libGLU-devel-9.0.0-4.el7.ppc64 mesa-libGLw-debuginfo-8.0.0-4.el7.ppc64 mesa-private-llvm-3.8.1-1.el7.ppc64 mesa-libOSMesa-11.2.2-2.20160614.el7.ppc64 mesa-dri-drivers-11.2.2-2.20160614.el7.ppc64 mesa-libEGL-devel-11.2.2-2.20160614.el7.ppc64 mesa-debuginfo-17.0.1-2.20170307.el7.ppc64 mesa-libgbm-11.2.2-2.20160614.el7.ppc64 mesa-libGLES-11.2.2-2.20160614.el7.ppc64 mesa-libgbm-devel-11.2.2-2.20160614.el7.ppc64 mesa-libGL-devel-11.2.2-2.20160614.el7.ppc64 mesa-libGLU-9.0.0-4.el7.ppc64 mesa-libGLU-debuginfo-9.0.0-4.el7.ppc64 mesa-libglapi-11.2.2-2.20160614.el7.ppc64 All works.
By old I mean of course 7.3 one.
(In reply to Tomas Pelka from comment #6) > Apr 04 16:03:45 ibm-p740-01-lp1.rhts.eng.bos.redhat.com gnome-shell[9146]: > JS ERROR: Exception in callback for signal: desktop-changed: TypeError: > this._applicationsButtons.forEach is not a function That's my fault - when backporting the patch in bug 1236601, I didn't double-check that I wasn't relying on newer mozjs functionality (Map.prototype.forEach was added in 25.0, gjs in RHEL uses 24.0). I'll use the original bug to fix this.
NOTE: This issue is ppc64 specific, output is normal on ppc64le. Will check s390x, as that is also big-endian arch.
We've just fixed our testbed on s390x and reproduced the same there. Thus this happens exclusively on big-endian arches and not on little-endians. So the endianism could relate to the issue.
So I assume the forEach problem is a separate issue and unrelated to the core problem, the black screen. if you do mutter --replace it's still black right?
(In reply to Ray Strode [halfline] from comment #16) > if you do mutter --replace it's still black right? Or log into the regular "GNOME" session instead of "GNOME Classic".
(In reply to Ray Strode [halfline] from comment #16) > So I assume the forEach problem is a separate issue and unrelated to the > core problem, the black screen. > > if you do mutter --replace it's still black right? That is correct, still black.
Created attachment 1270490 [details] better, but not adequate So I did a scratch build with the reversion mentioned in freedesktop.org 100613 and it helped quite a bit (no more black screen!), but still produced some misrendering. see above screenshot.
(In reply to Ray Strode [halfline] from comment #19) > Created attachment 1270490 [details] > better, but not adequate > > So I did a scratch build with the reversion mentioned in freedesktop.org > 100613 and it helped quite a bit (no more black screen!), but still produced > some misrendering. see above screenshot. This looks like channel swizzling confusion, since for 32-bit argb data if you're byte-swapped in the wrong place A and B get interchanged.
so i did some more investigation (bisecting) on this the last few days. It seems like comment 19 is an llvm regression caused by this commit: https://github.com/llvm-mirror/llvm/commit/76b12c4bf0bbc5c70def7b5d083a8a70547ea4e3
so worse comes to worse if we need a "right now" solution we can just ship the two reverts. I've tested that and it works okay. I'll let tstellar chime in on a potentially more right fix for llvm and Ben chime in on a potentially more right fix for mesa, but regardless we obviously can't ship 7.4 like this and we have potential path forward so devack+
Can you post the assembly/llvm IR output generated by the last good commit and also the commit that introduced the bug. The environment variable GALLIVM_DEBUG=ir,asm will dump this output for you. You will need to make sure these environment variables are set before you start gnome-shell.
Created attachment 1272993 [details] Vertex shader IR from before Mesa commit e827d917....
Created attachment 1273025 [details] Vertex shader IR from after Mesa commit e827d917... I typically set export MESA_DEBUG=verbose export MESA_GLSL="dump log uniform useprog errors" export GALLIVM_DEBUG="tgsi ir asm dumpbc" in my .bashrc. This attachment, vs-post-patch.ll, and its predecessor vs-pre-patch.ll are annotated with what I think are the significant differences.
Created attachment 1273086 [details] PPC64LE assembly code from vs-pre-patch.ll I've had a request to post the actual assembly code for the vertex shader programs, so here it is. First, the pre-patch assembly code from vs-pre-patch.ll, generated via % llc -mcpu=pwr8 -mattr=+altivec,+vsx vs-pre-patch.ll +altivec turns on FP vector support, while +vsx turns on the Vector-Scalar facility support.
Created attachment 1273087 [details] PPC64LE assembly code from vs-post-patch.ll As promised.... % llc -mcpu=pwr8 -mattr=+altivec,+vsx
Ben, are you able to reproduce the problem with newer versions of LLVM?
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:1865