Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1690082

Summary: gssproxy thread may block indefinitely inside epoll_wait() due to race with a second thread closing gpmctx->epollfd
Product: Red Hat Enterprise Linux 8 Reporter: Robbie Harwood <rharwood>
Component: gssproxyAssignee: Robbie Harwood <rharwood>
Status: CLOSED ERRATA QA Contact: anuja <amore>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.1CC: amore, dpal, dwysocha, fs-qe, ipa-qe, ksiddiqu, ndehadra
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: gssproxy-0.8.0-11.el8 Doc Type: Bug Fix
Doc Text:
(see rhel-7.7)
Story Points: ---
Clone Of: 1687899 Environment:
Last Closed: 2019-11-05 21:29:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1687899    
Bug Blocks: 1594286    

Description Robbie Harwood 2019-03-18 18:09:35 UTC
+++ This bug was initially created as a clone of Bug #1687899 +++

Description of problem:
There is a possible race condition inside gssproxy where one thread may call into epoll_wait(gpmctx->epollfd, ...) and while inside, a second thread may call close on (gpmctx->epollfd).  The thread inside epoll_wait is not woken up despite the second thread's close() operation but may hang inside epoll_wait().

We discovered this issue when studying a possible kernel bug where processes could hang indefinitely waiting for rpc.gssd upcall:
https://bugzilla.redhat.com/show_bug.cgi?id=1511706#c41

gpm_make_call() may release the mutex too early and as a result a thread race may be possible.

int gpm_make_call(int proc, union gp_rpc_arg *arg, union gp_rpc_res *res)
...
    /* grab the lock for the whole conversation */
    ret = gpm_grab_sock(gpmctx);
...    
    /* Send request, receive response with timeout */
    ret = gpm_send_recv_loop(gpmctx, send_buffer, send_length, &recv_buffer,
                             &recv_length);
    /* release the lock */
    gpm_release_sock(gpmctx);
    sockgrab = false;         // at this point some other thread could grab the lock and race with this one
...
done:
    gpm_timer_close(gpmctx);  // sets timerfd = -1 as seen in corefile
    gpm_epoll_close(gpmctx);  // sets epollfd = -1 as seen in corefile
...




Version-Release number of selected component (if applicable):
gssproxy-0.7.0-21.el7

How reproducible:
TBD - should be reproducible with delay inserted in gssproxy code

Steps to Reproduce:
See attached program which demonstrates the possibility of a hang inside epoll_wait()

Actual results:
gssproxy thread may hang inside epoll_wait() indefinitely

Expected results:
no indefinite hang inside epoll_wait() in gssproxy


Additional info:

Since this problem is so well defined and has a patch under testing, it makes sense to file this as a specific bug.

Comment 5 anuja 2019-08-12 13:08:37 UTC
Verified with sanity only run 
test-suite : upstream: test_integration/test_nfs.py

[1m============================= test session starts ==============================[0m
platform linux -- Python 3.6.8, pytest-3.4.2, py-1.5.3, pluggy-0.6.0 -- /usr/libexec/platform-python
cachedir: .pytest_cache
rootdir: /home/cloud-user, inifile:
plugins: sourceorder-0.5, multihost-3.0
[1mcollecting ... [0mcollected 4 items

test_nfs.py::TestNFS::test_prepare_users <- ../../usr/lib/python3.6/site-packages/ipatests/test_integration/test_nfs.py [32mPASSED[0m[36m [ 25%][0m
test_nfs.py::TestNFS::test_krb5_nfsd <- ../../usr/lib/python3.6/site-packages/ipatests/test_integration/test_nfs.py [32mPASSED[0m[36m [ 50%][0m
test_nfs.py::TestNFS::test_krb5_nfs_manual_configuration <- ../../usr/lib/python3.6/site-packages/ipatests/test_integration/test_nfs.py [32mPASSED[0m[36m [ 75%][0m
test_nfs.py::TestNFS::test_automount <- ../../usr/lib/python3.6/site-packages/ipatests/test_integration/test_nfs.py [32mPASSED[0m[36m [100%][0m

---------------- generated xml file: /home/cloud-user/junit.xml ----------------
[32m[1m========================== 4 passed in 682.39 seconds ==========================[0m

Comment 7 anuja 2019-08-13 07:42:48 UTC
Verified Using version:
python3-gssapi-1.5.1-5.el8.x86_64
mod_auth_gssapi-1.6.1-6.el8.x86_64
ipa-server-4.8.0-6.module+el8.1.0+3869+84a20141.x86_64
gssproxy-0.8.0-14.el8.x86_64

Comment 9 errata-xmlrpc 2019-11-05 21:29:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3515