Bug 508438

Summary: CRM 1927152 Incorrect results in gluUnProject and gluUnProject4 - incorrect matrix inversion (__gluInvertMatrixd)
Product: Red Hat Enterprise Linux 5 Reporter: Issue Tracker <tao>
Component: mesaAssignee: Adam Jackson <ajax>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.3CC: mgordon, mkrcmari, rkhadgar, spoyarek, tao, tjamrisk, vbenes
Target Milestone: rcKeywords: Patch, Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-02-21 06:48:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 726826    
Attachments:
Description Flags
patch based on upstream code
none
old/new functions comparison none

Description Issue Tracker 2009-06-27 08:30:17 UTC
Escalated to Bugzilla from IssueTracker

Comment 1 Issue Tracker 2009-06-27 08:30:19 UTC
Event posted on 18-06-2009 11:38am IST by spoyarek

Description of problem:

Base OS: RHEL-5.3

The matrix inversion function (__gluInvertMatrixd) in mesa-glu has an error, which leads to incorrect result with gluUnProject(). This is affecting a customer. Upstream bugzilla for the same:

http://bugs.freedesktop.org/show_bug.cgi?id=6748

How reproducible:

Always.

Steps to Reproduce:

1) Untar test case
2) Use the compileMe script to build test case
3) Execute test case

Actual results:

read z-buffer:0.795453

Expected results:

read z-buffer:0.795455

Additional info:

This was fixed upstream in two attempts:

http://cgit.freedesktop.org/mesa/mesa/commit/?id=f0a036bad91d5f281f8007e8cddefb3f648f1684
http://cgit.freedesktop.org/mesa/mesa/commit/?id=c2e1ee08b083638a7306e15ca48e9ea516d42dc4

Attached is a patch that combines the two. Customer has confirmed that the fix works for him.
This event sent from IssueTracker by rkhadgar  [Support Engineering Group]
 issue 308615

Comment 2 Issue Tracker 2009-06-27 08:30:21 UTC
Event posted on 18-06-2009 11:53am IST by spoyarek

The reason for second attempt:

http://bugs.freedesktop.org/show_bug.cgi?id=12269


Issue escalated to Support Engineering Group by: spoyarek.
spoyarek assigned to issue for Production Support (Pune).
Category set to: Applications::Bug Fix
Internal Status set to 'Waiting on SEG'
Status set to: Waiting on Tech

This event sent from IssueTracker by rkhadgar  [Support Engineering Group]
 issue 308615

Comment 3 ritz 2009-06-27 08:33:47 UTC
Created attachment 349637 [details]
patch based on upstream code

thanks to Siddesh for locating this patch.

Comment 6 RHEL Program Management 2009-11-06 19:27:09 UTC
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".

Comment 9 Adam Jackson 2011-08-17 18:41:41 UTC
3564086 build (dist-5E-qu-candidate, /cvs/dist:rpms/mesa/RHEL-5:mesa-6_5_1-7_9_el5) completed successfully

MODIFIED

Comment 11 Marian Krcmarik 2012-01-13 22:20:18 UTC
I followed steps to reproduce and verification failed for me, I compiled the test case from attachment and run it on:

# rpm -qa | egrep "mesa|freeglut"
freeglut-2.4.0-7.1.el5
mesa-libGL-devel-6.5.1-7.10.el5
mesa-libGL-6.5.1-7.10.el5
mesa-libGLU-6.5.1-7.10.el5
freeglut-2.4.0-7.1.el5
mesa-source-6.5.1-7.10.el5
freeglut-devel-2.4.0-7.1.el5
mesa-libGLU-6.5.1-7.10.el5
mesa-libGLU-devel-6.5.1-7.10.el5
freeglut-devel-2.4.0-7.1.el5
mesa-libGL-6.5.1-7.10.el5
mesa-libGLU-devel-6.5.1-7.10.el5

and result:

# ./gluUnProject.nvidia
freeglut (./gluUnProject.nvidia): Unable to create direct context rendering for window './gluUnProject.nvidia'
This may hurt performance.

./gluUnProject.nvidia - demonstrates use of matrix modes and stacks

Escape Key         - exit the program

read z-buffer:0.795453
read z-buffer:0.795453

Comment 12 Mark Gordon 2012-01-17 20:25:43 UTC
I'm seeing the same thing.  It's not clear to me whether this means that the fix isn't working as expected or that the reproducer isn't as valid as expected.

Ajax: any insights here?

Comment 13 Adam Jackson 2012-01-25 16:47:04 UTC
No idea honestly.  Possibly this varies by architecture, or based on whether you build with -ffast-math.  But that's kind of reaching.

Comment 15 Vladimir Benes 2012-02-02 11:37:50 UTC
ok .. so this is what I observed:

old mesa/new mesa

intel:
0.795455/0.795455

nv:
0.795453/0.795453

radeon:
0.795453/0.795453

nvidia 180.44 (this is what customer used)
0.795455/0.795455

nvidia 290.10 (newest one)
0.795455/0.795455

this looks like we are not affected by this bug anymore so here comes the question if we need the patch or it should be reverted. -ffast-math is not changing anything. Going to move to 5.3 to see if I can reproduce the issue there. 
I am using 01:00.0 VGA compatible controller [0300]: nVidia Corporation NV41 [Quadro FX 3450/4000 SDI] [10de:00cd] (rev a2)

Will try to write another reproducer using __gluInvertMatrixd function with values from https://bugs.freedesktop.org/show_bug.cgi?id=6748#c7 for example.. this should tell us if matrix inversion was done correctly or not. 
Stay tuned...

Comment 16 Vladimir Benes 2012-02-02 13:51:21 UTC
Created attachment 559055 [details]
old/new functions comparison

I've written reproducer that uses old and new code and comparing results with http://www.bluebit.gr/matrix-calculator/default.aspx results

as we can see there is a difference in cell [0][0] so I suspect that the previous reproducer is not valid as it counts the same values all the time.

I think this verification is sufficient.
-> VERIFIED

Comment 17 errata-xmlrpc 2012-02-21 06:48:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0288.html