Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 649810

Summary: Integer overflow for dashed lines longer than 46340
Product: Red Hat Enterprise Linux 5 Reporter: Olivier Fourdan <ofourdan>
Component: xorg-x11-serverAssignee: Adam Jackson <ajax>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: medium Docs Contact:
Priority: low    
Version: 5.5CC: kem, mgordon, spoyarek, tjamrisk, vbenes
Target Milestone: rcKeywords: Patch, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 649809 Environment:
Last Closed: 2012-02-21 03:11:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 649809, 649811    
Bug Blocks: 726826    
Attachments:
Description Flags
Reproducer program
none
Proposed patch none

Description Olivier Fourdan 2010-11-04 15:51:09 UTC 
Created attachment 457833 [details]
Reproducer program

+++ This bug was initially created as a clone of Bug #649809 +++

Description of problem:

Lines of length greater than 46340 can be drawn with one of the coordinates being negative. However for dashed lines, miPolyBuildPoly() overflows the "int" type when setting up edges for a section of the dashed line. This results in the dashed segments not being drawn at all.

Version-Release number of selected component (if applicable):

xorg-x11-server-1.7.7-26.el6

How reproducible:

Always

Steps to Reproduce:

1. Save attached reproducer program as line.c
2. Build the rproducer program with
   $ gcc -DBREAK -o line line.c -lX11
3. Run the test program in X11
   $ ./line
4. Right-click within the window created by the test application
  
Actual results:

No dotted line is displayed

Expected results:

A dotted line is shown

Additional info:

Remove the "-DBREAK"  from step #2 to get the expected result. The difference is that defining BREAK sets y2 to 31341 causing the overflow.

This has been reported upstream as bug 31093:

  https://bugs.freedesktop.org/show_bug.cgi?id=31093

The patch proposed by Siddhesh Poyarekar has been reviewed by Keith Packard:

  http://lists.x.org/archives/xorg-devel/2010-October/014559.html
Comment 1 Olivier Fourdan 2010-11-04 15:51:46 UTC 
Created attachment 457834 [details]
Proposed patch

Patch by Siddhesh Poyarekar proposed in upstream bug fd.o #31093
Comment 2 Matěj Cepl 2010-11-12 21:06:49 UTC 
Actually, on RHEL-5 it is even more complicated than on Fedora 14 and RHEL 6. While there the bug is reproducable with results exactly as described in the comment 0, it is slightly different here. When -DBREAK is present and I right-clicked on the window, there was some delay when nothing happened and then continuous (uninterrupted) line was drawn in the place where without -DBREAK interrupted line was.
Comment 3 Matěj Cepl 2010-11-13 19:51:39 UTC 
Hmm, next time I tried it didn't show anything like with other versions of RHEL.
Comment 5 RHEL Program Management 2011-05-31 14:38:41 UTC 
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.
Comment 8 Adam Jackson 2011-08-17 19:08:31 UTC 
3564178 build (dist-5E-qu-candidate, /cvs/dist:rpms/xorg-x11-server/RHEL-5:xorg-x11-server-1_1_1-48_82_el5) completed successfully

MODIFIED
Comment 11 errata-xmlrpc 2012-02-21 03:11:04 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0303.html