Bug 503796 - gdm-rh-security crashed by segfault when shutting down the X Windows System
gdm-rh-security crashed by segfault when shutting down the X Windows System
Status: CLOSED DUPLICATE of bug 658506
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: gdm (Show other bugs)
All Linux
high Severity medium
: rc
: 5.6
Assigned To: Ray Strode [halfline]
: Patch
Depends On:
Blocks: 502912 719046 668957
  Show dependency treegraph
Reported: 2009-06-02 14:24 EDT by Jeff Bastian
Modified: 2011-08-17 08:23 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2011-08-17 08:23:05 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jeff Bastian 2009-06-02 14:24:38 EDT
Description of problem:
When shutting down, rebooting, or switching to run-level 3, gdm occasionally segfaults instead of shutting down cleanly if the pcscd service is running.

/var/log/messages contains entries like:
Jun  1 17:23:14 sun-x6240-1 kernel: gdm-rh-security[18244]: segfault at 0000000000000000 rip 00002b401411419c rsp 000000004118eeb0 error 4

This doesn't seem to be a critical problem since gdm was shutting down anyway, however, if you're monitoring the logs for segfaults, this can raise alarms in the monitoring service.

Initial analysis from the customer:
   SCardUnload() of the pcsc-lite package operates as a destructor of the
   parent process when gdm-rh-security-token-helper ends, and the
   table(readerStates) is released. However, the segmentation fault happens
   because the child thread refers the released readerStates.

   Therefore, we think that exclusive control for readerStates is needed when
   readerStates is munmaped at SCardUnload() of the pcsc-lite package.
   It is also possible to evade the problem by ending
   gdm-rh-security-token-helper after the pcscd service is stopped.

Version-Release number of selected component (if applicable):

How reproducible:
fairly often

Steps to Reproduce:
1. init 3
2. (if necessary) chkconfig pcscd on; service pcscd start
3. run the following script as root
   while true; do
       echo "############## rep $COUNT ########"
       init 5
       sleep 10
       echo "runlevel: 5"
       echo "helper: `pgrep gdm-rh-security`"
       echo "pcscd: `pgrep pcscd`"

       init 3
       sleep 5
       echo "runlevel: 3"
       echo "helper: `pgrep gdm-rh-security`"
       echo "pcscd: `pgrep pcscd`"

       COUNT=`expr $COUNT + 1`
Actual results:
gdm will occasionally -- at least once an hour, possibly much more frequently -- crash with a segfault instead of shutting down cleanly

Expected results:
no segfaults

Additional info:
Comment 1 Jeff Bastian 2009-06-02 15:01:36 EDT
Created attachment 346299 [details]
core dump from /usr/libexec/gdm-rh-security-token-helper

(gdb) thread apply all bt

Thread 2 (process 4081):
#0  0x000000354be0d337 in close () from /lib64/libpthread.so.0
#1  0x00002ab048486cad in SCardUnload () at winscard_clnt.c:3495
#2  0x00002ab048485eff in __do_global_dtors_aux ()
   from /usr/lib64/libpcsclite.so.1
#3  0x0000000000000000 in ?? ()

Thread 1 (process 4094):
#0  SCardGetStatusChange (hContext=<value optimized out>,
    dwTimeout=<value optimized out>, rgReaderStates=<value optimized out>,
    cReaders=<value optimized out>) at winscard_clnt.c:1792
#1  0x00002ab04827f3ba in CKYCardContext_WaitForStatusChange (
    ctx=<value optimized out>, readers=<value optimized out>,
    readerCount=<value optimized out>, timeout=<value optimized out>)
    at cky_card.c:837
#2  0x00002ab0480692bc in SlotList::waitForSlotEvent ()
   from /usr/lib64/libcoolkeypk11.so
#3  0x00002ab0480617de in C_WaitForSlotEvent ()
   from /usr/lib64/libcoolkeypk11.so
#4  0x000000355e4580a8 in SECMOD_WaitForAnyTokenEvent (mod=0x117e1690,
    flags=0, latency=1000) at pk11util.c:1080
#5  0x0000000000404e17 in sc_security_token_monitor_worker_run (
    worker=0x117d1ea0) at securitytokenmonitor.c:1129
#6  0x000000354d248f24 in ?? ()
#7  0x0000000000000000 in ?? ()
Comment 2 Jeff Bastian 2009-06-02 15:16:09 EDT
Further analysis done by Takuma Umeya shows that thread 2 is in SCardUnload() which frees up the readerStates[] array:

3478 void DESTRUCTOR SCardUnload(void)
3479 {
3490                         SYS_PublicMemoryUnmap(readerStates[i], sizeof(READER_STATE));
3491                         readerStates[i] = NULL;
3495         SYS_CloseFile(mapAddr);

Meanwhile, thread 1 is in SCardGetStatusChange() and trying to read the readerStates[] array that was just freed:

1658 LONG SCardGetStatusChange(SCARDCONTEXT hContext, DWORD dwTimeout,
1659         LPSCARD_READERSTATE_A rgReaderStates, DWORD cReaders)
1660 {
1789         /* Get the initial reader count on the system */
1790         for (j=0; j < PCSCLITE_MAX_READERS_CONTEXTS; j++)
1791                 if ((readerStates[j])->readerID != 0)
1792                         currentReaderCount++;

gdm needs to wait for thread 1 to finish before calling SCardUnload()
Comment 10 RHEL Product and Program Management 2009-11-06 14:12:29 EST
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 12 RHEL Product and Program Management 2010-08-09 15:24:44 EDT
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.
Comment 16 Vladimir Benes 2011-08-17 08:23:05 EDT

*** This bug has been marked as a duplicate of bug 658506 ***

Note You need to log in before you can comment on or make changes to this bug.