Bug 1039126

Summary: vnc.so module causes Xorg crash with GNOME session
Product: Red Hat Enterprise Linux 7 Reporter: Tim Waugh <twaugh>
Component: tigervncAssignee: Tim Waugh <twaugh>
Status: CLOSED CURRENTRELEASE QA Contact: Alois Mahdal <amahdal>
Severity: unspecified Docs Contact:
Priority: high    
Version: 7.0CC: ajax, jscotka
Target Milestone: rcKeywords: Patch
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tigervnc-1.2.80-0.22.20130314svn5065.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 11:30:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
tigervnc12-xorg113-glx.patch (replacement) none

Description Tim Waugh 2013-12-06 17:41:35 UTC
Description of problem:
The vnc.so module causes a crash (glx-related?) when starting a GNOME session.

Version-Release number of selected component (if applicable):
tigervnc-1.2.80-0.20.20130314svn5065.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Install tigervnc-server, tigervnc-server-module, and xorg-x11-xdm (for easier testing)
2.systemctl disable gdm.service; systemctl enable xdm.service
2.As root, run "vncpasswd" and set a password.
3.Edit /etc/X11/xorg.conf.d/10-libvnc.conf and uncomment all the config lines (might need to change DefaultDepth to 24)
4.Restart
5.Log in.

Actual results:
Session does not start. Back to the xdm prompt.

Expected results:
Session starts.

Additional info:
If you try with gdm, the X server crashes repeatedly and this makes debugging tricky.

Comment 1 Tim Waugh 2013-12-06 17:56:56 UTC
Xorg: ../include/privates.h:122: dixGetPrivateAddr: Assertion `key->initialized' failed.

Program received signal SIGABRT, Aborted.
[...]
(gdb) bt
#0  0x0000003e3f035979 in raise () from /lib64/libc.so.6
#1  0x0000003e3f037088 in abort () from /lib64/libc.so.6
#2  0x0000003e3f02e8e6 in __assert_fail_base () from /lib64/libc.so.6
#3  0x0000003e3f02e992 in __assert_fail () from /lib64/libc.so.6
#4  0x00007f0ac6db41c0 in dixGetPrivateAddr (key=<optimized out>, 
    key=<optimized out>, privates=<optimized out>) at ../include/privates.h:122
#5  0x00007f0ac6dcb23f in dixGetPrivateAddr (key=<optimized out>, 
    key=<optimized out>, privates=<optimized out>) at glxscreens.c:194
#6  dixLookupPrivate (key=0x7f0ac6fe79a0 <glxScreenPrivateKeyRec>, 
    privates=<optimized out>) at ../include/privates.h:165
#7  glxGetScreen (pScreen=<optimized out>) at glxscreens.c:193
#8  0x00007f0ac6dc5088 in validGlxScreen (screen=<optimized out>, 
    pGlxScreen=pGlxScreen@entry=0x7fffc8057508, err=<optimized out>, 
    client=0x19c57e0) at glxcmds.c:63
#9  0x00007f0ac6dc7f4a in validGlxScreen (err=<synthetic pointer>, 
    pGlxScreen=0x7fffc8057508, screen=<optimized out>, client=0x19c57e0)
    at glxcmds.c:2406
#10 __glXDisp_QueryServerString (cl=<optimized out>, pc=0x19b8a30 "\232\023\003")
    at glxcmds.c:2405
#11 0x00007f0ac6dc9b70 in __glXDispatch (client=<optimized out>) at glxext.c:601
#12 0x00000000004372ef in Dispatch () at dispatch.c:434
#13 0x000000000043b10a in dix_main (argc=1, argv=0x7fffc8057788, 
    envp=<optimized out>) at main.c:294
#14 0x0000003e3f021af5 in __libc_start_main () from /lib64/libc.so.6
#15 0x0000000000426651 in _start ()

Comment 4 Tim Waugh 2013-12-09 16:46:38 UTC
Doesn't happen with tigervnc-1.3.0 (fedora/master) rebuilt for rhel-7.0.

Comment 5 Tim Waugh 2013-12-09 17:17:19 UTC
Created attachment 834407 [details]
tigervnc12-xorg113-glx.patch (replacement)

Here is the fix, found by comparing differences between our 1.2.80 and 1.3.0 packages.

Comment 7 Alois Mahdal 2014-02-17 12:43:07 UTC
Verified with tigervnc-1.2.80-0.22.20130314svn5065.el7

With old version, I get crash cycle (I tested without xdm since it's not available in RHEL7).  After installing the new build, GDM starts and I can log in normally.

Comment 8 Ludek Smid 2014-06-13 11:30:10 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.