Bug 638553 - Destroying a torn-off menu with a submenu mapped causes a crash in OpenMotif
Summary: Destroying a torn-off menu with a submenu mapped causes a crash in OpenMotif
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: openmotif
Version: 5.5
Hardware: All
OS: Linux
urgent
high
Target Milestone: rc
: ---
Assignee: Thomas Woerner
QA Contact: Filip Holec
URL:
Whiteboard:
Depends On:
Blocks: 712073
TreeView+ depends on / blocked
 
Reported: 2010-09-29 10:34 UTC by Olivier Fourdan
Modified: 2018-11-14 18:52 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, a check that would limit removing a callback to valid windows while the focus is reset was missing in the code. Consequently, destroying a torn-off menu with a submenu mapped caused the application to terminate unexpectedly. With this update, the underlying source code has been modified to ensure that the focus is reset for valid windows only and destroying a torn-off menu with a submenu mapped now works as expected.
Clone Of:
Environment:
Last Closed: 2013-10-01 00:33:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Proposed patch - needs review (662 bytes, patch)
2010-09-29 10:46 UTC, Olivier Fourdan
no flags Details | Diff
Upstream patch (1.47 KB, patch)
2011-05-02 09:25 UTC, J.H.M. Dassen (Ray)
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
MotifZone 1535 0 None None None Never
Red Hat Product Errata RHBA-2013:1355 0 normal SHIPPED_LIVE openmotif bug fix and enhancement update 2013-09-30 21:12:33 UTC

Description Olivier Fourdan 2010-09-29 10:34:16 UTC
Description of problem:

When destroying a torn-off popup menu to rebuild it while that menu has another submenu mapped can lead to a segmentation fault in OpenMotif.

Version-Release number of selected component (if applicable):

OpenMotif-2.3.1

How reproducible:

Always

Steps to Reproduce:
1. Save and build the attached test case program:

   $ cc -o testcase -g tear_off_menu.c -I/usr/X11R6/include -lX11 -lXm -lc

2. Run the program

   $ ./testcase

3. In the main window, right click to open the menu and tear it off by clicking on the "----" at the top

4. select one of the sub-menu

5. While the sub-menu is still mapped and there's an active grab on the pointer, right click again on the main window to re-open the menu
 
6. Navigate in the sub-menu again
  
Actual results:

The program will terminate with a segmentation fault

Expected results:

The program should not crash

Additional info:

Destroying the menu is required in the case of our customer because in the real application, the content of the menu and sub-menus actually depends on the location of the click.

Comment 1 Olivier Fourdan 2010-09-29 10:38:26 UTC
Created attachment 450442 [details]
Reproducer program provided by our customer

This is the reproducer code as provided by our customer.

Comment 2 Olivier Fourdan 2010-09-29 10:46:25 UTC
Created attachment 450443 [details]
Proposed patch - needs review

The backtrace shows that the crash occurs while removing the existing callbacks:

The backtrace varies, but the common code path includes:

Program terminated with signal 11, Segmentation fault.
#0  0x000000321fc1399c in FetchInternalList (widget=0xfbe5860, name=<value optimized out>) at Callback.c:92
92		if (quark == (*offsets)->xrm_name) {
(gdb) bt
#0  0x000000321fc1399c in FetchInternalList (widget=0xfbe5860, name=<value optimized out>) at Callback.c:92
#1  0x000000321fc14312 in XtRemoveCallback (widget=0xfbe5860, name=0x606599 "destroyCallback", 
    callback=0x321e1823f0 <InvalidateOldFocus>, closure=0xfc095d8) at Callback.c:322
#2  0x000000321e185437 in _XmMenuFocus (w=0xfc273c0, operation=<value optimized out>, _time=1319101) at RCMenu.c:1084
#3  0x000000321e0dadf1 in PopupSharedMenuShell (cbwid=0xfc27020, smwid=0xfc27750, event=0x0) at MenuShell.c:1249
#4  0x000000321e08d05c in Popup (cb=0xfc27020, event=0x0, doCascade=<value optimized out>) at CascadeB.c:1824
#5  _XmCascadingPopup (cb=0xfc27020, event=0x0, doCascade=<value optimized out>) at CascadeB.c:1641
#6  0x000000321fc2dc77 in DoOtherSources (app=0xfbb40e0) at NextEvent.c:1142
#7  0x000000321fc2ddc3 in XtAppNextEvent (app=0xfbb40e0, event=0x7fffdeb59ba0) at NextEvent.c:1234
#8  0x000000321fc20d6d in XtAppMainLoop (app=0xfbb40e0) at Event.c:1548
#9  0x0000000000400d63 in main (argc=1, argv=0x7fffdeb59dc8) at tear_off_menu.c:63
(gdb) f 1
#1  0x000000321fc14312 in XtRemoveCallback (widget=0xfbe5860, name=0x606599 "destroyCallback", 
    callback=0x321e1823f0 <InvalidateOldFocus>, closure=0xfc095d8) at Callback.c:322
322	    callbacks = FetchInternalList(widget, name);
(gdb) p *widget
$1 = {core = {self = 0xfbe5ab0, widget_class = 0x300af52a38, parent = 0xfbd4840, xrm_name = 835, being_destroyed = 1 '\001', 
    destroy_callbacks = 0xfc1a3b0, constraints = 0x0, x = 203, y = 102, width = 34, height = 130, border_width = 0, 
    managed = 0 '\000', sensitive = 1 '\001', ancestor_sensitive = 1 '\001', event_table = 0xfbfc130, tm = {translations = 0x0, 
      proc_table = 0x0, current_state = 0x0, lastEventTime = 0}, accelerators = 0x0, border_pixel = 0, border_pixmap = 2, 
    popup_list = 0x0, num_popups = 0, name = 0xfbcc4d2 "", screen = 0xfbb7b60, colormap = 32, window = 77594731, depth = 24, 
    background_pixel = 12895428, background_pixmap = 2, visible = 1 '\001', mapped_when_managed = 1 '\001'}}
(gdb) f 2
#2  0x000000321e185437 in _XmMenuFocus (w=0xfc273c0, operation=<value optimized out>, _time=1319101) at RCMenu.c:1084
1084			XtRemoveCallback(mst->RC_menuFocus.oldWidget, 

in frame #2, we see that XtRemoveCallback() is called on the so called "oldWidget" and that widget has its "being_destroyed" field set.

The proposed patch here just adds an additional check for oldWidget->core.being_destroyed and does not do the XtRemoveCallback() if set.

Although that patch seems to fix the issue, it would require additional review.

Comment 6 RHEL Program Management 2011-01-11 20:55:47 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.

Comment 7 RHEL Program Management 2011-01-11 23:16:59 UTC
This request was erroneously denied for the current release of
Red Hat Enterprise Linux.  The error has been fixed and this
request has been re-proposed for the current release.

Comment 12 J.H.M. Dassen (Ray) 2011-05-02 09:25:02 UTC
Created attachment 496196 [details]
Upstream patch

Fix that has been committed to upstream CVS.

Comment 17 RHEL Program Management 2011-05-05 14:24:56 UTC
Development Management has reviewed and declined this request.  You may appeal
this decision by reopening this request.

Comment 18 Phil Knirsch 2011-05-05 14:40:29 UTC
Accidentally closed, reopening.

Sorry for the error.

Thanks & regards, Phil

Comment 22 Tomas Capek 2011-07-15 12:33:56 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, a check that would limit removing a callback to valid windows while the focus is reset was missing in the code. Consequently, destroying a torn-off menu with a submenu mapped caused the application to terminate unexpectedly. With this update, the underlying source code has been modified to ensure that the focus is reset for valid windows only and destroying a torn-off menu with a submenu mapped now works as expected.

Comment 29 errata-xmlrpc 2013-10-01 00:33:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1355.html


Note You need to log in before you can comment on or make changes to this bug.