Created attachment 411011 [details]
proposed patch by spoyarek

Description of problem:

When an X client running on a big endian system (e.g. SPARC/Solaris) does a XineramaQueryScreens(), the X server crashes.

Version-Release number of selected component (if applicable):

xorg-x11-server-Xnest-1.1.1-48.76.el5

How reproducible:

Always

Steps to Reproduce:
1. From a Red Hat Enterprise Linux X server, login via ssh to a Solaris/SPARC machine
2. Run an application that does a XineramaQueryScreens()
  
Actual results:

The X server crashes

Expected results:

The X server does not crash

Additional info:

This is due to the swap of bytes that is done when dealing with a big endian client.

In Procxf86XineramaQueryScreens() from hw/xfree86/modes/xf86Xinerama.c :

268 int
269 Procxf86XineramaQueryScreens(ClientPtr client)
270 {
271     xXineramaQueryScreensReply  rep;
272     ScreenPtr   pScreen = screenInfo.screens[xf86_XINERAMA_SCREEN];
273     ScrnInfoPtr         scrn = xf86Screens[pScreen->myNum];
274     xf86CrtcConfigPtr   xf86_config = XF86_CRTC_CONFIG_PTR(scrn);
275 
276     REQUEST_SIZE_MATCH(xXineramaQueryScreensReq);
277 
278     rep.type = X_Reply;
279     rep.sequenceNumber = client->sequence;
280     rep.number = xf86XineramaScreenCount (pScreen);
281     rep.length = rep.number * sz_XineramaScreenInfo >> 2;
282     if(client->swapped) {
283         register int n;
284         swaps(&rep.sequenceNumber, n);
285         swapl(&rep.length, n);
286         swapl(&rep.number, n);xf86XineramaScreenCount
287     }
288     WriteToClient(client, sizeof(xXineramaQueryScreensReply), (char *)&rep);
289 
290     if(rep.number) {
291         xXineramaScreenInfo scratch;
292         int i;
293 
294         for(i = 0; i < rep.number; i++) {
295             xf86CrtcPtr crtc = xf86_config->crtc[i];

So the value of rep.number we use line 294 has its byte swapped from line 286 causing "i" to go way beyond the value initially returned by xf86XineramaScreenCount() line 280, causing the crash of the X server.

The fix proposed by Siddhesh Poyarekar is to save the actual value of "rep.number" before performing the byte swap so it can be reused later in the code.