Bug 1241227

Summary: stalled gdm-session-worker after a bunch of tests
Product: Red Hat Enterprise Linux 7 Reporter: Vladimir Benes <vbenes>
Component: gdmAssignee: Ray Strode [halfline] <rstrode>
Status: CLOSED ERRATA QA Contact: Desktop QE <desktop-qa-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.2CC: angystardust, mdomonko, tlavigne
Target Milestone: rcKeywords: Regression, TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: gdm-3.14.2-7.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-19 07:16:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vladimir Benes 2015-07-08 19:07:23 UTC
Description of problem:
I can see a load of stalled gdm-session-worker processes after let's say hundreds of gdm service start/stops. This happens when we start and stop tests often during tests. In some particular case it prevents gdm to start.

[root@qe-dell-ovs5-vm-23 gnome-initial-setup]# ps aux|grep gdm-session-worker
root       889  0.0  0.4 323876  7768 ?        Sl   13:13   0:04 gdm-session-worker [pam/gdm-password]
root      1940  0.1  0.5 325044 10420 ?        Sl   12:39   0:10 gdm-session-worker [pam/gdm-password]
root      2363  0.0  0.5 324044 10044 ?        Sl   13:15   0:04 gdm-session-worker [pam/gdm-password]
root      3418  0.1  0.5 324924 10404 ?        Sl   12:41   0:09 gdm-session-worker [pam/gdm-password]
root      4387  0.1  0.6 325584 11684 ?        Sl   12:23   0:13 gdm-session-worker [pam/gdm-password]
root      5903  0.1  0.6 325756 11768 ?        Sl   12:24   0:13 gdm-session-worker [pam/gdm-password]
root      5992  0.2  0.7 326456 14328 ?        Sl   11:50   0:23 gdm-session-worker [pam/gdm-password]
root      7472  0.2  0.7 326640 14452 ?        Sl   11:52   0:22 gdm-session-worker [pam/gdm-password]
root      8764  0.0  0.3 323716  7144 ?        Sl   13:22   0:03 gdm-session-worker [pam/gdm-password]
root      9730  0.1  0.5 324652  9648 ?        Sl   12:48   0:08 gdm-session-worker [pam/gdm-password]
root     10328  0.0  0.3 323856  7128 ?        Sl   13:23   0:03 gdm-session-worker [pam/gdm-password]
root     11232  0.1  0.5 324760  9760 ?        Sl   12:49   0:08 gdm-session-worker [pam/gdm-password]
root     13654  0.1  0.7 326080 13548 ?        Sl   11:58   0:20 gdm-session-worker [pam/gdm-password]
root     15181  0.1  0.7 326420 13712 ?        Sl   12:00   0:20 gdm-session-worker [pam/gdm-password]
root     15378  0.2  0.8 327064 16396 ?        Sl   11:26   0:30 gdm-session-worker [pam/gdm-password]
root     16867  0.0  0.3 323628  6376 ?        Sl   13:31   0:02 gdm-session-worker [pam/gdm-password]
root     16895  0.2  0.8 327420 16756 ?        Sl   11:28   0:30 gdm-session-worker [pam/gdm-password]
root     17540  0.0  0.4 324416  8996 ?        Sl   12:56   0:06 gdm-session-worker [pam/gdm-password]
root     18444  0.0  0.3 323684  6428 ?        Sl   13:32   0:02 gdm-session-worker [pam/gdm-password]
root     19044  0.0  0.4 324520  9168 ?        Sl   12:58   0:06 gdm-session-worker [pam/gdm-password]
root     21397  0.1  0.6 326068 13024 ?        Sl   12:06   0:18 gdm-session-worker [pam/gdm-password]
root     22855  0.1  0.6 326244 13148 ?        Sl   12:08   0:18 gdm-session-worker [pam/gdm-password]
root     23098  0.2  0.8 327036 15856 ?        Sl   11:34   0:28 gdm-session-worker [pam/gdm-password]
root     24362  0.0  0.0 112660   936 pts/0    R+   14:58   0:00 grep --color=auto gdm-session-worker
root     24585  0.2  0.9 327348 17920 ?        Sl   11:36   0:28 gdm-session-worker [pam/gdm-password]
root     25391  0.0  0.4 324140  8612 ?        Sl   13:04   0:05 gdm-session-worker [pam/gdm-password]
root     26606  0.1  0.5 325192 11076 ?        Sl   12:31   0:12 gdm-session-worker [pam/gdm-password]
root     26943  0.0  0.4 324192  8572 ?        Sl   13:06   0:05 gdm-session-worker [pam/gdm-password]
root     28069  0.1  0.5 325416 11148 ?        Sl   12:32   0:11 gdm-session-worker [pam/gdm-password]
root     29081  0.1  0.6 325764 12480 ?        Sl   12:15   0:16 gdm-session-worker [pam/gdm-password]
root     30584  0.1  0.6 326236 12396 ?        Sl   12:16   0:15 gdm-session-worker [pam/gdm-password]
root     30755  0.2  0.9 327148 17164 ?        Sl   11:42   0:25 gdm-session-worker [pam/gdm-password]
root     32256  0.2  0.7 327076 15000 ?        Sl   11:44   0:25 gdm-session-worker [pam/gdm-password]

any idea why is this happening?

test itself looks like this:

userdel -rf test
sleep 2

service gdm start
export DISPLAY=:0
sleep 5

behave tests/features -kt $1 -f plain -o /tmp/report_$TEST.log -f plain; rc=$?


systemctl stop gdm.service
sleep 3
for i in $(ps aux|grep gdm-session-worker |awk {'print $2'}); do kill -9 $i; done
rm -rf /run/gnome-initial-setup*
userdel -rf johnnie

useradd -m test
echo "redhat" |passwd --stdin test

Version-Release number of selected component (if applicable):
pam-1.1.8-12.el7.x86_64
gdm-3.14.2-4.el7.x86_64
gnome-shell-3.14.4-10.el7.x86_64

How reproducible:
always

Comment 2 Ray Strode [halfline] 2015-07-14 15:35:44 UTC
right

Comment 3 Ray Strode [halfline] 2015-07-15 17:09:11 UTC
should be all set now.  This needed a few fixes backported from the 3.16 and master branches

Comment 5 Vladimir Benes 2015-07-17 11:30:58 UTC
It looks far better, I've removed workarounds and tests are still running (100+ gdm restarts). Thanks!

Comment 7 errata-xmlrpc 2015-11-19 07:16:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2216.html

Comment 8 Angelo Lisco 2017-12-16 16:10:05 UTC
Sorry for the noise but it seems that the issue isn't fixed at all or maybe there was a regression in the last vesion of gdm (gdm-3.26.2.1).
I know this bug report is about RHEL but I suppose that sooner or later gdm-2.26 will hit the RHEL repos.

The issue is always reproducible by locking the screen, slide the screen to fill your password in the text-box, press [ESC] to undo the filling of the password.

As you can see by grepping on the process list on my system, it's easy to create a local denial-of-service as a malicious user just by following the above sequence over and over again. Sooner or later i suppose a "too many open files" condition can be reached so that the user cannot login on the system.

$ ps -ef | grep [g]dm-session-worker
root      3278 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3298 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3316 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3334 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3367 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3385 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3406 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3423 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root      3489 16037  0 16:40 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     15672 15668  0 01:13 ?        00:00:00 gdm-session-worker [pam/gdm-launch-environment]
root     16037 15668  0 01:13 ?        00:00:01 gdm-session-worker [pam/gdm-fingerprint]
root     21990 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     21991 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-fingerprint]
root     22011 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22029 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22048 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22068 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22089 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22108 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22126 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22143 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22162 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22186 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22204 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22222 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22239 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22262 16037  0 17:00 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     22917 15668  0 01:18 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     25673 16037  0 01:20 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     25692 16037  0 01:20 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     25712 16037  0 01:20 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     25735 16037  0 01:20 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     25755 16037  0 01:20 ?        00:00:00 gdm-session-worker [pam/gdm-password]
root     25784 16037  0 01:20 ?        00:00:00 gdm-session-worker [pam/gdm-password]