Bug 588346 - XineramaQueryScreens() from an X client on a big endian machine cause the Xserver to crash
Summary: XineramaQueryScreens() from an X client on a big endian machine cause the Xse...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: xorg-x11-server
Version: 5.5
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Adam Jackson
QA Contact: Desktop QE
URL:
Whiteboard:
Depends On:
Blocks: 726826
TreeView+ depends on / blocked
 
Reported: 2010-05-03 14:15 UTC by Olivier Fourdan
Modified: 2018-11-27 21:50 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-02-21 03:10:39 UTC
Target Upstream Version:


Attachments (Terms of Use)
proposed patch by spoyarek@redhat.com (1.12 KB, patch)
2010-05-03 14:15 UTC, Olivier Fourdan
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2012:0303 0 normal SHIPPED_LIVE Low: xorg-x11-server security and bug fix update 2012-02-21 07:24:37 UTC

Description Olivier Fourdan 2010-05-03 14:15:01 UTC
Created attachment 411011 [details]
proposed patch by spoyarek@redhat.com

Description of problem:

When an X client running on a big endian system (e.g. SPARC/Solaris) does a XineramaQueryScreens(), the X server crashes.

Version-Release number of selected component (if applicable):

xorg-x11-server-Xnest-1.1.1-48.76.el5

How reproducible:

Always

Steps to Reproduce:
1. From a Red Hat Enterprise Linux X server, login via ssh to a Solaris/SPARC machine
2. Run an application that does a XineramaQueryScreens()
  
Actual results:

The X server crashes

Expected results:

The X server does not crash

Additional info:

This is due to the swap of bytes that is done when dealing with a big endian client.

In Procxf86XineramaQueryScreens() from hw/xfree86/modes/xf86Xinerama.c :

268 int
269 Procxf86XineramaQueryScreens(ClientPtr client)
270 {
271     xXineramaQueryScreensReply  rep;
272     ScreenPtr   pScreen = screenInfo.screens[xf86_XINERAMA_SCREEN];
273     ScrnInfoPtr         scrn = xf86Screens[pScreen->myNum];
274     xf86CrtcConfigPtr   xf86_config = XF86_CRTC_CONFIG_PTR(scrn);
275 
276     REQUEST_SIZE_MATCH(xXineramaQueryScreensReq);
277 
278     rep.type = X_Reply;
279     rep.sequenceNumber = client->sequence;
280     rep.number = xf86XineramaScreenCount (pScreen);
281     rep.length = rep.number * sz_XineramaScreenInfo >> 2;
282     if(client->swapped) {
283         register int n;
284         swaps(&rep.sequenceNumber, n);
285         swapl(&rep.length, n);
286         swapl(&rep.number, n);xf86XineramaScreenCount
287     }
288     WriteToClient(client, sizeof(xXineramaQueryScreensReply), (char *)&rep);
289 
290     if(rep.number) {
291         xXineramaScreenInfo scratch;
292         int i;
293 
294         for(i = 0; i < rep.number; i++) {
295             xf86CrtcPtr crtc = xf86_config->crtc[i];

So the value of rep.number we use line 294 has its byte swapped from line 286 causing "i" to go way beyond the value initially returned by xf86XineramaScreenCount() line 280, causing the crash of the X server.

The fix proposed by Siddhesh Poyarekar is to save the actual value of "rep.number" before performing the byte swap so it can be reused later in the code.

Comment 2 RHEL Program Management 2010-09-29 16:47:24 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.

Comment 4 RHEL Program Management 2011-05-31 15:35:45 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.

Comment 7 Adam Jackson 2011-08-18 14:09:18 UTC
Built in xorg-x11-server-1.1.1-48.84.el5

MODIFIED

Comment 11 errata-xmlrpc 2012-02-21 03:10:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0303.html


Note You need to log in before you can comment on or make changes to this bug.