Bug 1445423

Summary: blue shadows on ppc64 and s390x
Product: Red Hat Enterprise Linux 7 Reporter: Ray Strode [halfline] <rstrode>
Component: mesa-private-llvmAssignee: Tom Stellard <tstellar>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: urgent Docs Contact:
Priority: high    
Version: 7.4CC: airlied, bcrocker, bmilar, desktop-qa-list, fmuellner, jkoten, jstodola, mcepl, mkrajnak, mnewsome, rstrode, salmy, tpelka, tstellar, vhumpa
Target Milestone: rcKeywords: Regression, TestBlocker
Target Release: 7.4   
Hardware: ppc64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1438891 Environment:
Last Closed: 2017-08-01 16:03:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1438891, 1456040    
Bug Blocks: 1395724, 1446209    
Attachments:
Description Flags
screenshot with llvm 4
none
List of piglit tests fixed by reverting 76b12c4bf0bbc5c70def7b5d083a8a70547ea4e3 none

Description Ray Strode [halfline] 2017-04-25 16:04:16 UTC
+++ This bug was initially created as a clone of Bug #1438891 +++
--- Additional comment from Ray Strode [halfline] on 2017-04-10 09:20 EDT ---

So I did a scratch build with the reversion mentioned in freedesktop.org 100613 and it helped quite a bit (no more black screen!), but still produced some misrendering. see above screenshot.

--- Additional comment from Adam Jackson on 2017-04-11 14:36:30 EDT ---

(In reply to Ray Strode [halfline] from comment #19)
> Created attachment 1270490 [details]
> better, but not adequate
> 
> So I did a scratch build with the reversion mentioned in freedesktop.org
> 100613 and it helped quite a bit (no more black screen!), but still produced
> some misrendering. see above screenshot.

This looks like channel swizzling confusion, since for 32-bit argb data if you're byte-swapped in the wrong place A and B get interchanged.

--- Additional comment from Ray Strode [halfline] on 2017-04-14 09:41:02 EDT ---

so i did some more investigation (bisecting) on this the last few days.  It seems like comment 19 is an llvm regression caused by this commit:

https://github.com/llvm-mirror/llvm/commit/76b12c4bf0bbc5c70def7b5d083a8a70547ea4e3

--- Additional comment from Ray Strode [halfline] on 2017-04-14 17:19:30 EDT ---

so worse comes to worse if we need a "right now" solution we can just ship the two reverts.  I've tested that and it works okay.

I'll let tstellar chime in on a potentially more right fix for llvm and Ben chime in on a potentially more right fix for mesa, but regardless we obviously can't ship 7.4 like this and we have potential path forward so devack+

--- Additional comment from Tom Stellard on 2017-04-20 09:51:26 EDT ---

Can you post the assembly/llvm IR output generated by the last good commit and also the commit that introduced the bug.  The environment variable GALLIVM_DEBUG=ir,asm will dump this output for you.

You will need to make sure these environment variables are set before you start gnome-shell.

--- Additional comment from Tom Stellard on 2017-04-24 05:35:41 EDT ---

Ben, are you able to reproduce the problem with newer versions of LLVM?

--- Additional comment from Ben Crocker on 2017-04-24 10:31:18 EDT ---

Tom, please clarify....
Ray says the problem was introduced somewhere between 3.8 & 3.9,
and reminds me that he identified the specific commit;
which newer version(s) are you asking about?  Was there a 3.9.x after
3.9?

I know there is a 4.0; is there anything more recent?  (LLVM site
says 4.0.0 is the latest release, so I guess I'm asking about 
whatever the next release is going to be.)

--- Additional comment from Steve Almy on 2017-04-24 10:48:52 EDT ---

changing from alpha to beta blocker, based on email feedback from EPM.

--- Additional comment from Ray Strode [halfline] on 2017-04-24 16:07:47 EDT ---

we may want to clone this bug into two issues.

To be clear there's the mesa problem in comment 0 and the llvm problem in comment 19.

--- Additional comment from Tom Stellard on 2017-04-24 20:57:32 EDT ---

(In reply to Ben Crocker from comment #30)
> Tom, please clarify....
> Ray says the problem was introduced somewhere between 3.8 & 3.9,
> and reminds me that he identified the specific commit;
> which newer version(s) are you asking about?  Was there a 3.9.x after
> 3.9?
> 
> I know there is a 4.0; is there anything more recent?  (LLVM site
> says 4.0.0 is the latest release, so I guess I'm asking about 
> whatever the next release is going to be.)

I meant testing with 4.0 or LLVM trunk, to see if this has been fixed upstream already.

Comment 1 Ben Crocker 2017-04-26 17:48:09 UTC
This may be interesting:

I added some glReadPixels calls to glxgears; on PPC64LE, I got back the expected values: mostly red for the big gear; mostly green for the one to the right, and mostly blue for the one on top.

Interestingly, when I built & ran the modified glxgears on my PPC64 BE system (with Roland's e827d917 patch reverted so the geometry is right), glReadPixels read back the same expected values, but the colors still looked wrong on the
display.

Comment 2 Ray Strode [halfline] 2017-04-27 13:58:41 UTC
Created attachment 1274676 [details]
screenshot with llvm 4

So I did a build against the tip of release_40 branch and the problem is still present

Comment 8 Tom Stellard 2017-05-04 17:51:03 UTC
Created attachment 1276401 [details]
List of piglit tests fixed by reverting 76b12c4bf0bbc5c70def7b5d083a8a70547ea4e3

Piglit runs on ppc64 system with fedora25.  Test result 'bad' is with f25's LLVM 3.9.1, the result 'fix' is LLVM 3.9.1 from upstream with 76b12c4bf0bbc5c70def7b5d083a8a70547ea reverted.

Comment 10 Ben Crocker 2017-05-08 19:27:32 UTC
Ray and I were just running a gnome-shell test with LLVM 3.9 installed
AND your Mesa patch, and we can confirm that this problem is fixed.

Comment 11 Tom Stellard 2017-05-10 16:26:48 UTC
I submitted a new brew build with this fix: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=556531

Comment 15 errata-xmlrpc 2017-08-01 16:03:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:1865