Bug 508438 - CRM 1927152 Incorrect results in gluUnProject and gluUnProject4 - incorrect matrix inversion (__gluInvertMatrixd)
CRM 1927152 Incorrect results in gluUnProject and gluUnProject4 - incorrect m...
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: mesa (Show other bugs)
All Linux
medium Severity medium
: rc
: ---
Assigned To: Adam Jackson
Desktop QE
: Patch, Triaged
Depends On:
Blocks: 726826
  Show dependency treegraph
Reported: 2009-06-27 04:30 EDT by Issue Tracker
Modified: 2012-02-21 01:48 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2012-02-21 01:48:33 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
patch based on upstream code (4.53 KB, patch)
2009-06-27 04:33 EDT, ritz
no flags Details | Diff
old/new functions comparison (6.19 KB, text/plain)
2012-02-02 08:51 EST, Vladimir Benes
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
FreeDesktop.org 6748 None None None Never

  None (edit)
Description Issue Tracker 2009-06-27 04:30:17 EDT
Escalated to Bugzilla from IssueTracker
Comment 1 Issue Tracker 2009-06-27 04:30:19 EDT
Event posted on 18-06-2009 11:38am IST by spoyarek

Description of problem:

Base OS: RHEL-5.3

The matrix inversion function (__gluInvertMatrixd) in mesa-glu has an error, which leads to incorrect result with gluUnProject(). This is affecting a customer. Upstream bugzilla for the same:


How reproducible:


Steps to Reproduce:

1) Untar test case
2) Use the compileMe script to build test case
3) Execute test case

Actual results:

read z-buffer:0.795453

Expected results:

read z-buffer:0.795455

Additional info:

This was fixed upstream in two attempts:


Attached is a patch that combines the two. Customer has confirmed that the fix works for him.
This event sent from IssueTracker by rkhadgar  [Support Engineering Group]
 issue 308615
Comment 2 Issue Tracker 2009-06-27 04:30:21 EDT
Event posted on 18-06-2009 11:53am IST by spoyarek

The reason for second attempt:


Issue escalated to Support Engineering Group by: spoyarek.
spoyarek assigned to issue for Production Support (Pune).
Category set to: Applications::Bug Fix
Internal Status set to 'Waiting on SEG'
Status set to: Waiting on Tech

This event sent from IssueTracker by rkhadgar  [Support Engineering Group]
 issue 308615
Comment 3 ritz 2009-06-27 04:33:47 EDT
Created attachment 349637 [details]
patch based on upstream code

thanks to Siddesh for locating this patch.
Comment 6 RHEL Product and Program Management 2009-11-06 14:27:09 EST
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 9 Adam Jackson 2011-08-17 14:41:41 EDT
3564086 build (dist-5E-qu-candidate, /cvs/dist:rpms/mesa/RHEL-5:mesa-6_5_1-7_9_el5) completed successfully

Comment 11 Marian Krcmarik 2012-01-13 17:20:18 EST
I followed steps to reproduce and verification failed for me, I compiled the test case from attachment and run it on:

# rpm -qa | egrep "mesa|freeglut"

and result:

# ./gluUnProject.nvidia
freeglut (./gluUnProject.nvidia): Unable to create direct context rendering for window './gluUnProject.nvidia'
This may hurt performance.

./gluUnProject.nvidia - demonstrates use of matrix modes and stacks

Escape Key         - exit the program

read z-buffer:0.795453
read z-buffer:0.795453
Comment 12 Mark Gordon 2012-01-17 15:25:43 EST
I'm seeing the same thing.  It's not clear to me whether this means that the fix isn't working as expected or that the reproducer isn't as valid as expected.

Ajax: any insights here?
Comment 13 Adam Jackson 2012-01-25 11:47:04 EST
No idea honestly.  Possibly this varies by architecture, or based on whether you build with -ffast-math.  But that's kind of reaching.
Comment 15 Vladimir Benes 2012-02-02 06:37:50 EST
ok .. so this is what I observed:

old mesa/new mesa




nvidia 180.44 (this is what customer used)

nvidia 290.10 (newest one)

this looks like we are not affected by this bug anymore so here comes the question if we need the patch or it should be reverted. -ffast-math is not changing anything. Going to move to 5.3 to see if I can reproduce the issue there. 
I am using 01:00.0 VGA compatible controller [0300]: nVidia Corporation NV41 [Quadro FX 3450/4000 SDI] [10de:00cd] (rev a2)

Will try to write another reproducer using __gluInvertMatrixd function with values from https://bugs.freedesktop.org/show_bug.cgi?id=6748#c7 for example.. this should tell us if matrix inversion was done correctly or not. 
Stay tuned...
Comment 16 Vladimir Benes 2012-02-02 08:51:21 EST
Created attachment 559055 [details]
old/new functions comparison

I've written reproducer that uses old and new code and comparing results with http://www.bluebit.gr/matrix-calculator/default.aspx results

as we can see there is a difference in cell [0][0] so I suspect that the previous reproducer is not valid as it counts the same values all the time.

I think this verification is sufficient.
Comment 17 errata-xmlrpc 2012-02-21 01:48:33 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.