Bug 1032379

Summary: [rpc.gssd][gssproxy] enable gssproxy, run mount rpc.gssd segfault happen
Product: Red Hat Enterprise Linux 7 Reporter: JianHong Yin <jiyin>
Component: nfs-utilsAssignee: Steve Dickson <steved>
Status: CLOSED CURRENTRELEASE QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: urgent Docs Contact:
Priority: medium    
Version: 7.0   
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-17 05:14:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description JianHong Yin 2013-11-20 04:35:30 UTC
Description of problem:

================================================================================
[23:15:21 root@ /mnt/tests/CoreOS/nfs-utils/krb5-test/base]# service rpcgssd start; service rpcgssd status
Redirecting to /bin/systemctl start  rpcgssd.service
Redirecting to /bin/systemctl status  rpcgssd.service
nfs-secure.service - Secure NFS
   Loaded: loaded (/usr/lib/systemd/system/nfs-secure.service; enabled)
   Active: active (running) since Tue 2013-11-19 22:59:50 EST; 15min ago
 Main PID: 747 (rpc.gssd)
   CGroup: /system.slice/nfs-secure.service
           └─747 /usr/sbin/rpc.gssd

Nov 19 22:59:50 intel-mahobay-01.rhts.eng.nay.redhat.com systemd[1]: Started Secure NFS.
Nov 19 23:15:21 intel-mahobay-01.rhts.eng.nay.redhat.com systemd[1]: Started Secure NFS.
:: [   PASS   ] :: Running 'service rpcgssd start; service rpcgssd status' (Expected 0, got 0)
--------------------------------------------------------------------------------

================================================================================
[23:15:53 root@ /mnt/tests/CoreOS/nfs-utils/krb5-test/base]# timeLimitRun 30 mount -vvv -o vers=4 $SERVER_nfs:/usr/share  /nfsmp
{WARN} cmd[mount -vvv -o vers=4 hp-dl580g7-03.rhts.eng.nay.redhat.com:/usr/share /nfsmp] timeOut!
/mnt/tests/kernel/Library/base/lib.sh: line 135: 17524 Terminated              sh -c ": $cmdId; $*"
:: [   FAIL   ] :: Running 'timeLimitRun 30 mount -vvv -o vers=4 $SERVER_nfs:/usr/share  /nfsmp' (Expected 0, got 143)
--------------------------------------------------------------------------------

================================================================================
[23:16:24 root@ /mnt/tests/CoreOS/nfs-utils/krb5-test/base]# tail -n30 /var/log/messages
Nov 19 23:11:36 intel-mahobay-01 logger: 2013-11-19 23:11:36 /usr/bin/rhts-test-runner.sh 3136 720 hearbeat...
Nov 19 23:12:36 intel-mahobay-01 logger: 2013-11-19 23:12:36 /usr/bin/rhts-test-runner.sh 3136 780 hearbeat...
Nov 19 23:13:36 intel-mahobay-01 logger: 2013-11-19 23:13:36 /usr/bin/rhts-test-runner.sh 3136 840 hearbeat...
Nov 19 23:14:36 intel-mahobay-01 systemd: Starting Cleanup of Temporary Directories...
Nov 19 23:14:36 intel-mahobay-01 systemd-tmpfiles: [/usr/lib/tmpfiles.d/systemd.conf:27] Unknown file type 'm'.
Nov 19 23:14:36 intel-mahobay-01 systemd-tmpfiles: [/usr/lib/tmpfiles.d/systemd.conf:28] Unknown file type 'm'.
Nov 19 23:14:36 intel-mahobay-01 systemd-tmpfiles: [/usr/lib/tmpfiles.d/systemd.conf:29] Unknown file type 'm'.
Nov 19 23:14:36 intel-mahobay-01 systemd-tmpfiles: [/usr/lib/tmpfiles.d/systemd.conf:30] Unknown file type 'm'.
Nov 19 23:14:36 intel-mahobay-01 systemd: systemd-tmpfiles-clean.service: main process exited, code=exited, status=1/FAILURE
Nov 19 23:14:36 intel-mahobay-01 systemd: Failed to start Cleanup of Temporary Directories.
Nov 19 23:14:36 intel-mahobay-01 systemd: Unit systemd-tmpfiles-clean.service entered failed state.
Nov 19 23:14:36 intel-mahobay-01 logger: 2013-11-19 23:14:36 /usr/bin/rhts-test-runner.sh 3136 900 hearbeat...
Nov 19 23:15:17 intel-mahobay-01 yum[16908]: Installed: 1:telnet-0.17-57.el7.x86_64
Nov 19 23:15:19 intel-mahobay-01 systemd: Stopped firewalld.service.
Nov 19 23:15:20 intel-mahobay-01 systemd: Starting GSSAPI Proxy Daemon...
Nov 19 23:15:20 intel-mahobay-01 systemd: Started GSSAPI Proxy Daemon.
Nov 19 23:15:21 intel-mahobay-01 systemd: Started Secure NFS.
Nov 19 23:15:21 intel-mahobay-01 kernel: [  945.652469] FS-Cache: Loaded
Nov 19 23:15:21 intel-mahobay-01 kernel: [  945.672329] FS-Cache: Netfs 'nfs' registered for caching
Nov 19 23:15:21 intel-mahobay-01 kernel: [  945.673959] Key type dns_resolver registered
Nov 19 23:15:21 intel-mahobay-01 kernel: [  945.686763] NFS: Registering the id_resolver key type
Nov 19 23:15:21 intel-mahobay-01 kernel: [  945.686776] Key type id_resolver registered
Nov 19 23:15:21 intel-mahobay-01 kernel: [  945.686777] Key type id_legacy registered
Nov 19 23:15:21 intel-mahobay-01 kernel: [  945.691347] rpc.gssd[17099]: segfault at 0 ip 00007f9812d63f0a sp 00007fffabe086e8 error 4 in libc-2.17.so[7f9812c32000+1b6000]
Nov 19 23:15:21 intel-mahobay-01 rpc.gssd[747]: WARNING: forked child was killed with signal 11
        <<<<<<<<------------- rpc.gssd segfault

Nov 19 23:15:36 intel-mahobay-01 logger: 2013-11-19 23:15:36 /usr/bin/rhts-test-runner.sh 3136 960 hearbeat...
Nov 19 23:15:53 intel-mahobay-01 kernel: [  976.903291] NFS: nfs4_discover_server_trunking unhandled error -512. Exiting with error EIO
Nov 19 23:15:53 intel-mahobay-01 kernel: [  976.948321] rpc.gssd[17530]: segfault at 0 ip 00007f9812d63f0a sp 00007fffabe086e8 error 4 in libc-2.17.so[7f9812c32000+1b6000]
Nov 19 23:15:53 intel-mahobay-01 rpc.gssd[747]: WARNING: forked child was killed with signal 11
Nov 19 23:16:24 intel-mahobay-01 kernel: [ 1008.232525] NFS: nfs4_discover_server_trunking unhandled error -512. Exiting with error EIO
:: [   PASS   ] :: Running 'tail -n30 /var/log/messages' (Expected 0, got 0)

Version-Release number of selected component (if applicable):
------------------------------------------------
TimeInfo  : 2013-11-19 23:02:01
CaseName  : /CoreOS/nfs-utils/krb5-test/base
$HOSTNAME : intel-mahobay-01.rhts.eng.nay.redhat.com
DistroInfo: RedHatEnterpriseWorkstation 7.0
kernelInfo: Linux intel-mahobay-01.rhts.eng.nay.redhat.com 3.10.0-52.el7.x86_64 #1 SMP Fri Nov 15 14:46:15 EST 2013 x86_64 x86_64 x86_64 GNU/Linux
packageInfo
	libnfsidmap-0.25-7.el7.x86_64
	libsss_idmap-1.11.2-1.el7.x86_64
	nfs-utils-1.2.9-0.3.el7.x86_64
------------------------------------------------

How reproducible:


Steps to Reproduce:
1. enable gssproxy
   start rpcgssd service
2. mount remote dir exported with sec=krb5
3. 

Actual results:
segfault happened in rpc.gssd.

Expected results:
works fine

Additional info:
   the same test without gssproxy, works fine.

Comment 3 Steve Dickson 2013-11-21 20:59:49 UTC
I'm not able to reproduce this with the latest kernel, nfs-utils
or gssproxy packages: 

kernel-3.10.0-53.el7.x86_64
nfs-utils-1.2.9-0.3.el7
gssproxy-0.3.0-3.el7

Could you please retest with those?

Also are you setting USE_GSS_PROXY=1 in /etc/sysconfig/nfs?
I tried setting it and not setting and still was not able
to reproduce this.

Comment 4 JianHong Yin 2013-11-22 05:06:30 UTC
(In reply to Steve Dickson from comment #3)
> I'm not able to reproduce this with the latest kernel, nfs-utils
> or gssproxy packages: 
> 
> kernel-3.10.0-53.el7.x86_64
> nfs-utils-1.2.9-0.3.el7
> gssproxy-0.3.0-3.el7
> 
> Could you please retest with those?
> 
> Also are you setting USE_GSS_PROXY=1 in /etc/sysconfig/nfs?
> I tried setting it and not setting and still was not able
> to reproduce this.

yes I set USE_GSS_PROXY="yes";
but now I can not reproduce that; I will try more test.