Bug 1266564

Summary: rpc.gssd is hangs in the gssproxy code
Product: Red Hat Enterprise Linux 7 Reporter: Steve Dickson <steved>
Component: gssproxyAssignee: Robbie Harwood <rharwood>
Status: CLOSED ERRATA QA Contact: Namita Soman <nsoman>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: eguan, fs-qe, nsoman, rharwood, ssorce, steved, tlavigne, yoyang
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-19 09:30:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1264999    
Attachments:
Description Flags
EINTR fix none

Description Steve Dickson 2015-09-25 15:49:30 UTC 
Description of problem:
Due to some recent changes to rpc.gssd, secure
NFS mount can hang in the gssproxy code. 

The changes to the rpc.gssd was to instead of
forking on every upcall, only for on non-root
upcalls. This means upcalls for mounts no 
longer forks. 

This bug fix is needed for bz 1264999 (which is a RHEL7.2 blocker)

Version-Release number of selected component (if applicable):
gssproxy-0.4.1-6.el7

How reproducible:
100%

Steps to Reproduce:
1. Change rpc.gssd to only for on non-root upcalls
2. do 3 secure NFS mounts at the same time, a number of times

Additional info:
The stack trace of the hung rpc.gssd 
#0  0x00007f45568ba980 in __read_nocancel () from /lib64/libc.so.6
#1  0x00007f45552b74a7 in gpm_make_call () from /usr/lib64/gssproxy/proxymech.so
#2  0x00007f45552b39ed in gpm_acquire_cred () from /usr/lib64/gssproxy/proxymech.so
#3  0x00007f45552b80e9 in gssi_acquire_cred () from /usr/lib64/gssproxy/proxymech.so
#4  0x00007f45572beb62 in gss_add_cred_from () from /lib64/libgssapi_krb5.so.2
#5  0x00007f45572bf02f in gss_acquire_cred_from () from /lib64/libgssapi_krb5.so.2
#6  0x00007f45572bf1e4 in gss_acquire_cred () from /lib64/libgssapi_krb5.so.2
#7  0x00007f455794ffe3 in gssd_acquire_krb5_cred (gss_cred=gss_cred@entry=0x7ffd279651e0)
    at krb5_util.c:1372
#8  0x00007f45579520c9 in limit_krb5_enctypes (sec=sec@entry=0x7ffd279651d0) at krb5_util.c:1433
#9  0x00007f455794dad9 in create_auth_rpc_client (clp=clp@entry=0x7f45583ffaf0, tgtname=tgtname@entry=0x0, 
    clnt_return=clnt_return@entry=0x7ffd27965710, auth_return=auth_return@entry=0x7ffd27965668, 
    uid=uid@entry=0, cred=cred@entry=0x0, authtype=0) at gssd_proc.c:878
#10 0x00007f455794f094 in krb5_use_machine_creds (clp=clp@entry=0x7f45583ffaf0, uid=uid@entry=0, 
    tgtname=tgtname@entry=0x0, service=service@entry=0x7f45583fe6c0 "*", 
    rpc_clnt=rpc_clnt@entry=0x7ffd27965710) at gssd_proc.c:1100
#11 0x00007f455794f40e in process_krb5_upcall (clp=clp@entry=0x7f45583ffaf0, uid=0, fd=16, 
    tgtname=tgtname@entry=0x0, service=service@entry=0x7f45583fe6c0 "*") at gssd_proc.c:1222
#12 0x00007f455794fcf3 in handle_gssd_upcall (clp=0x7f45583ffaf0) at gssd_proc.c:1413
#13 0x00007f455794d63e in scan_poll_results (ret=1) at gssd_main_loop.c:85
#14 gssd_poll (nfds=<optimized out>, fds=<optimized out>) at gssd_main_loop.c:201
#15 gssd_run () at gssd_main_loop.c:258
#16 0x00007f455794c10a in main (argc=<optimized out>, argv=0x7ffd27965b28) at gssd.c:216
Comment 2 Steve Dickson 2015-09-25 16:44:07 UTC 
On almost, if not, every upcall these gssproxy messages are being logged:

(OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
gssproxy[659]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
Comment 6 Robbie Harwood 2015-09-28 23:56:08 UTC 
Created attachment 1078086 [details]
EINTR fix
Comment 7 Robbie Harwood 2015-09-29 00:02:29 UTC 
Need ACKs for this fix.
Comment 10 Steve Dickson 2015-09-30 11:38:16 UTC 
The nfs-utils bits are already to go... committed/built/errata
so please make sure this makes into 7.2. If it does not all 
secure NFS mounts will be broken.
Comment 14 errata-xmlrpc 2015-11-19 09:30:57 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2298.html