Bug 1229607

Summary: nfs-ganesha: unexporting a volume fails and nfs-ganesha process coredumps
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Saurabh <saujain>
Component: nfs-ganeshaAssignee: Soumya Koduri <skoduri>
Status: CLOSED ERRATA QA Contact: storage-qa-internal <storage-qa-internal>
Severity: urgent Docs Contact:
Priority: high    
Version: rhgs-3.1CC: asrivast, mzywusko, nlevinki, skoduri, vagarwal
Target Milestone: ---   
Target Release: RHGS 3.1.0   
Hardware: x86_64   
OS: All   
Whiteboard:
Fixed In Version: nfs-ganesha-2.2.0-3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-29 04:59:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1202842    
Attachments:
Description Flags
coredump of nfs-ganesha process
none
sosreport of nfs9 none

Description Saurabh 2015-06-09 08:56:29 UTC 
Description of problem:
I tried to execute the case mentioned in BZ 1220999, this is about exporting a volume when selinux is enforced
Now, the case is tried with selinux enforced. 
I tried to export the volume manually, using dbus commands. This is successful done.
Now, I tried to unexport the volume and it fails, a coredump for nfs-ganesha is generated.

Version-Release number of selected component (if applicable):
glusterfs-3.7.0-3.el6rhs.x86_64
nfs-ganesha-gluster-2.2.0-2.el6rhs.x86_64

How reproducible:
always

Steps to Reproduce:
1.unexport the volume manually using the command,
dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:1

Actual results:
step1, core generated,
bt,
#0  0x00007f3f554e1e7b in export_release () from /usr/lib64/ganesha/libfsalgluster.so.4.2.0
Missing separate debuginfos, use: debuginfo-install dbus-libs-1.2.24-8.el6_6.x86_64 glibc-2.12-1.166.el6.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-42.el6.x86_64 libacl-2.2.49-6.el6.x86_64 libattr-2.4.44-7.el6.x86_64 libblkid-2.17.2-12.18.el6.x86_64 libcap-2.16-5.5.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libuuid-2.17.2-12.18.el6.x86_64 nfs-utils-lib-1.1.5-11.el6.x86_64 openssl-1.0.1e-39.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) 
(gdb) 
(gdb) bt
#0  0x00007f3f554e1e7b in export_release () from /usr/lib64/ganesha/libfsalgluster.so.4.2.0
#1  0x0000000000508578 in free_export_resources ()
#2  0x000000000051a2cd in free_export ()
#3  0x000000000051b759 in put_gsh_export ()
#4  0x000000000051c8f1 in gsh_export_removeexport ()
#5  0x000000000052ed50 in dbus_message_entrypoint ()
#6  0x0000003c5e41cefe in ?? () from /lib64/libdbus-1.so.3
#7  0x0000003c5e410b4c in dbus_connection_dispatch () from /lib64/libdbus-1.so.3
#8  0x0000003c5e410dd9 in ?? () from /lib64/libdbus-1.so.3
#9  0x000000000052f913 in gsh_dbus_thread ()
#10 0x0000003c5b407a51 in start_thread () from /lib64/libpthread.so.0
#11 0x0000003c5b0e896d in clone () from /lib64/libc.so.6
(gdb) quit

/var/log/audit/audit.logs

type=ANOM_ABEND msg=audit(1433849974.332:803): auid=0 uid=0 gid=0 ses=1 subj=unconfined_u:system_r:initrc_t:s0 pid=3637 comm="ganesha.nfsd" sig=11
type=USER_ACCT msg=audit(1433850001.272:804): user pid=3696 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_ACQ msg=audit(1433850001.272:805): user pid=3696 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=LOGIN msg=audit(1433850001.283:806): pid=3696 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=34
type=USER_START msg=audit(1433850001.289:807): user pid=3696 uid=0 auid=0 ses=34 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_DISP msg=audit(1433850001.371:808): user pid=3696 uid=0 auid=0 ses=34 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=USER_END msg=audit(1433850001.371:809): user pid=3696 uid=0 auid=0 ses=34 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'

Expected results:
the volume should get unexported without fail and without killing nfs-ganesha 

Additional info:
earlier to this, I had tried cli command "gluster volume set <vol-name> ganesha.enable off" and this also failed.
Comment 2 Saurabh 2015-06-09 08:57:23 UTC 
Created attachment 1036644 [details]
coredump of nfs-ganesha process
Comment 3 Saurabh 2015-06-09 09:24:18 UTC 
Created attachment 1036663 [details]
sosreport of nfs9
Comment 4 Saurabh 2015-06-10 06:12:24 UTC 
Updated the setup based on recommendations given in,
https://bugzilla.redhat.com/show_bug.cgi?id=1220999#c9

I tried to unexport a volume using the cli i.e. gluster volume set <volname> ganesha.enable off and the unexport still fails.
Comment 5 Soumya Koduri 2015-06-10 12:41:01 UTC 
We have a fix for this issue - 
https://review.gerrithub.io/#/c/236129/

Will include this fix in the next NFS-Ganesha RPM build
Comment 7 Saurabh 2015-06-17 08:52:08 UTC 
[root@nfs11 ~]# gluster volume set vol0 ganesha.enable on
volume set: success
[root@nfs11 ~]# showmount -e localhost
Export list for localhost:
/vol0 (everyone)


[root@nfs11 ~]# gluster volume set vol0 ganesha.enable off
volume set: success
[root@nfs11 ~]# showmount -e localhost
Export list for localhost:
[root@nfs11 ~]# 
[root@nfs11 ~]# 
[root@nfs11 ~]# 
[root@nfs11 ~]# rpm -qa | grep nfs-ganesha
nfs-ganesha-2.2.0-3.el6rhs.x86_64
nfs-ganesha-gluster-2.2.0-3.el6rhs.x86_64
Comment 8 errata-xmlrpc 2015-07-29 04:59:56 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1495.html