Bug 442510 - PATCH: glXChooseVisual behavior change causes SDL app using a stencil buffer to fail
PATCH: glXChooseVisual behavior change causes SDL app using a stencil buffer ...
Product: Fedora
Classification: Fedora
Component: xorg-x11-drv-ati (Show other bugs)
All Linux
low Severity low
: ---
: ---
Assigned To: Dave Airlie
Fedora Extras Quality Assurance
: 439682 (view as bug list)
Depends On:
Blocks: f9-x-blocker
  Show dependency treegraph
Reported: 2008-04-15 04:46 EDT by Hans de Goede
Modified: 2008-04-23 11:13 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-04-22 14:27:17 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
PATCH: glXChooseVisual behavior change causes SDL app using a stencil buffer to fail (744 bytes, patch)
2008-04-15 04:46 EDT, Hans de Goede
no flags Details | Diff
PATCH: prefer glxvisuals with stencil buffer for default visuals (1.64 KB, patch)
2008-04-18 18:51 EDT, Hans de Goede
no flags Details | Diff

External Trackers
Tracker ID Priority Status Summary Last Updated
FreeDesktop.org 15515 None None None Never

  None (edit)
Description Hans de Goede 2008-04-15 04:46:42 EDT
This commit:

Adds seperate visual and fbconfigs list to the GLX __GLXscreenConfigsRec struct,
where there used to be one list with both concatenated. 

It also changes various functions to use either the visual list or the fbcondig
list. Reading through various SGI manpages describing these functions these
changes seem correct.

However some applications / library depend on the old behavior, so this change
is causing various issues.

glXChooseVisual() used to check both real visuals and fbconfig "pseudo" visuals
to find a visual matching the requested attrributes, but now it only checks the
real visuals, which causes issues with SDL applications as SDL uses
glXChooseVisual() to get a visual and when the application and thus SDL requests
a stencilbuffer this fails, as only fbconfigs seem to offer stencilbuffers and
not the real visuals.

One could argue this is a bug in SDL that needs fixing, but this code has been
unchanged and worked in SDL for a long time, and it not only works with mesa but
also with other (nvidia, ati) opengl implementations.

So I would rather see this behavioral change reverted. The attached patch
restores the checking of both real visuals and fbconfigs in glXChooseVisual(),
fixing stencil buffer using SDL apps.

Note 1: I suspect that there are many other applications which will get bitten
by this behavioral change too, which is why I believe this should be fixed at
the mesa level.

Note 2: These changes cause other issues too, I'll fill a seperate bug for those.

Note 3: I would very much appreciate it if someone could reply to this quickly
(not fix, but reply) as the Fedora-9 release is in 2 weeks and I really want to
see this fixed before then.

Note 4: To reproduce the problem do: "yum install supertuxkart" and then run
"supertuxkart" note how it doesn't work, with older mesa and/or this patch it
does work.

Note 5: Also submitted upstream as:
Comment 1 Hans de Goede 2008-04-15 04:46:42 EDT
Created attachment 302423 [details]
PATCH: glXChooseVisual behavior change causes SDL app using a stencil buffer to fail
Comment 2 Hans de Goede 2008-04-15 14:51:33 EDT
krh, adding you to the CC as I just noticed the commit causing (exposing) this
issue was done by you. Any input on howto move forward with this would be much
appreciated, currently I'm thinking about either:
* use the patch attached here
* patch applications to use / also try glXChooseFBConfig()

Comment 3 Hans de Goede 2008-04-18 06:22:42 EDT
Okay, upstream has made the following comment with regards to this bug:
"It just needs to be fixed to include stencil bits in the visuals when

Some more investigation from my side has learned that on my intel graphics
machine not only the fbconfigs but also the normal visuals report stencil bits
(and this makes stencil using SDL apps work), here is the relevant output of

3 GLX Visuals
   visual  x  bf lv rg d st colorbuffer ax dp st accumbuffer  ms  cav
 id dep cl sp sz l  ci b ro  r  g  b  a bf th cl  r  g  b  a ns b eat
0x21 24 tc  0 32  0 r  y  .  8  8  8  8  0 24  8  0  0  0  0  0 0 None
0x22 24 dc  0 32  0 r  y  .  8  8  8  8  0 24  8  0  0  0  0  0 0 None
0x60 32 tc  0 32  0 r  .  .  8  8  8  8  0  0  0  0  0  0  0  0 0 None

Where on my ati r300, x86_64 system, the stcl column always reports 0 for the
visuals, and only reports stcl bits in the fbconfigs.

I've investigated the mesa code, and the reported stencil bits are gotten from
the X-server, and I assume through the X-server from the X-server driver, so I'm
changing the components and assignee to match.

Also including airlied in the CC, as he is our ATI expert. I'm willing to take a
shot at fixing this / to test any fixes, but I will need some pointers to get me
Comment 4 Hans de Goede 2008-04-18 18:51:43 EDT
Created attachment 302960 [details]
PATCH: prefer glxvisuals with stencil buffer for default visuals

The problem sits in GL/glx/glxscreens.c of the server.

There the first fbconfig which has a depthbuffer > 0  and doublebuf is
choosen when attaching fbconfigs to the standard Xvisuals, indepenent of
stencil bits, this happens to work ok on intel as there all fbconfigs with a
depthbuffer > 0 also have stencil bits.

This patch fixes this by first trying to get a fbconfig for default X visuals
with both stencilbuf, depthbuf and doublebuffering, and if that fails fallback
to trying to get one with only a depthbuf and doublebuffering.

Patch also submitted upstream:
Comment 5 Hans de Goede 2008-04-18 18:53:07 EDT
*** Bug 439682 has been marked as a duplicate of this bug. ***
Comment 6 Hans de Goede 2008-04-19 03:32:52 EDT

Perhaps the fbconfig for standard Xvisual choosing code should also be modified
to not choose any fbconfigs which have any Caveats, but that would be deviating
from the current behavior (where it doesn't care about Caveats) so I don't know
if this is a good idea, and certainly it is not a good idea this late in the F-9
Comment 7 Adam Jackson 2008-04-22 14:27:17 EDT
Fixed in  Thanks!
Comment 8 Kristian Høgsberg 2008-04-23 11:13:11 EDT
Yup, the patch looks good, thanks Hans.

Note You need to log in before you can comment on or make changes to this bug.