Description of problem: As reported in the upstream -https://github.com/nfs-ganesha/nfs-ganesha/issues/130#issuecomment-267903225, when there are multiple export entries with same parent volume and one export path is subset of another, the object handles are merged by md-cache across different export entries. Because of this, unexporting one entry shall result in SEGV when those handles are accessed via another share mount. Version-Release number of selected component (if applicable): 3.2 How reproducible: Always Steps to Reproduce: 1. Create 2+ shares using the same gluster volume 2. Delete all shares in any order (except opposite from creation). 3. Ganesha segaults. Actual results: NFS-Ganesha process crashes Expected results: There should not be any crash. Additional info:
Patch merged upstream - https://review.gerrithub.io/#/c/306836/
Tested with build # rpm -qa | grep ganesha nfs-ganesha-debuginfo-2.4.4-10.el7rhgs.x86_64 glusterfs-ganesha-3.8.4-31.el7rhgs.x86_64 nfs-ganesha-2.4.4-10.el7rhgs.x86_64 nfs-ganesha-gluster-2.4.4-10.el7rhgs.x86_64 Following are the observation- 1.Export the sub dir with dbus command ->Ganesha process goes to failed state. While Export subdir with ganesha service restart on all node ->Successfully exports the sub-dir #dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/var/run/gluster/shared_storage/nfs-ganesha/exports string:"EXPORT(Path=/ganeshavol5/mani3)" Error org.freedesktop.DBus.Error.NoReply: Message did not receive a reply (timeout by message bus) ganesha.log- 03/07/2017 18:51:18 : epoch 96d00000 : dhcp42-125.lab.eng.blr.redhat.com : ganesha.nfsd-12527[dbus_heartbeat] unregister_fsal :FSAL :CRIT :Unregister FSAL GLUSTER with non-zero refcount=2 03/07/2017 18:51:18 : epoch 96d00000 : dhcp42-125.lab.eng.blr.redhat.com : ganesha.nfsd-12527[dbus_heartbeat] glusterfs_unload :FSAL :CRIT :FSAL Gluster unable to unload. Dying ... 2.Volume stop/delete does not remove the volume export entry from showmount -e (when the sub dir are exported with dbus command) but mount on client gets failed. # showmount -e Export list for dhcp42-125.lab.eng.blr.redhat.com: /ganeshavol5 (everyone) /ganeshavol5/mani2 (everyone) /ganeshavol5/mani3 (everyone) # gluster v stop ganeshavol5 Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y volume stop: ganeshavol5: success [root@dhcp42-125 exports]# showmount -e Export list for dhcp42-125.lab.eng.blr.redhat.com: /ganeshavol5 (everyone) /ganeshavol5/mani2 (everyone) /ganeshavol5/mani3 (everyone) [root@dhcp42-125 exports]# gluster v status ganeshavol5 Volume ganeshavol5 is not started # gluster v delete ganeshavol5 Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y volume delete: ganeshavol5: success [root@dhcp42-125 exports]# showmount -e Export list for dhcp42-125.lab.eng.blr.redhat.com: /ganeshavol5 (everyone) /ganeshavol5/mani2 (everyone) /ganeshavol5/mani3 (everyone) #mount -t nfs -o vers=4 10.70.42.42:/ganeshavol5 /mnt/mani/ mount.nfs: Remote I/O error @Soumya can you comment on the above observations related to sub dir mount
In comment #7 dbus command used was incomplete. Verified this bug on # rpm -qa | grep ganesha nfs-ganesha-gluster-2.4.4-16.el7rhgs.x86_64 nfs-ganesha-2.4.4-16.el7rhgs.x86_64 glusterfs-ganesha-3.8.4-34.el7rhgs.x86_64 Steps: 1.Create 2+ shares of the same volume # WARNING : Using Gluster CLI will overwrite manual # changes made to this file. To avoid it, edit the # file and run ganesha-ha.sh --refresh-config. EXPORT{ Export_Id = 2; Path = "/ganeshavol3"; FSAL { name = GLUSTER; hostname="localhost"; volume="ganeshavol3"; } Access_type = RW; Disable_ACL = true; Squash="No_root_squash"; Pseudo="/ganeshavol3"; Protocols = "3", "4" ; Transports = "UDP","TCP"; SecType = "sys"; } EXPORT{ Export_Id = 3; Path = "/ganeshavol3/dir1"; FSAL { name = GLUSTER; hostname="localhost"; volume="ganeshavol3"; volpath="/dir1"; } Access_type = RW; Disable_ACL = true; Squash="No_root_squash"; Pseudo="/dir1"; Protocols = "3", "4" ; Transports = "UDP","TCP"; SecType = "sys"; } EXPORT{ Export_Id = 4; Path = "/ganeshavol3/dir2"; FSAL { name = GLUSTER; hostname="localhost"; volume="ganeshavol3"; volpath="/dir2"; } Access_type = RW; Disable_ACL = true; Squash="No_root_squash"; Pseudo="/dir2"; Protocols = "3", "4" ; Transports = "UDP","TCP"; SecType = "sys"; } EXPORT{ Export_Id = 5; Path = "/ganeshavol3/dir3"; FSAL { name = GLUSTER; hostname="localhost"; volume="ganeshavol3"; volpath="/dir3"; } Access_type = RW; Disable_ACL = true; Squash="No_root_squash"; Pseudo="/dir3"; Protocols = "3", "4" ; Transports = "UDP","TCP"; SecType = "sys"; } ---------- [root@dhcp42-125 exports]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol3.conf string:"EXPORT(Path=/ganeshavol3/dir1)" method return sender=:1.36 -> dest=:1.154 reply_serial=2 string "1 exports added" [root@dhcp42-125 exports]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol3.conf string:"EXPORT(Path=/ganeshavol3/dir2)" method return sender=:1.36 -> dest=:1.155 reply_serial=2 string "1 exports added" [root@dhcp42-125 exports]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.AddExport string:/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol3.conf string:"EXPORT(Path=/ganeshavol3/dir3)" method return sender=:1.36 -> dest=:1.158 reply_serial=2 string "1 exports added" [root@dhcp42-125 exports]# showmount -e Export list for dhcp42-125.lab.eng.blr.redhat.com: /ganeshavol3 (everyone) /ganeshavol3/dir1 (everyone) /ganeshavol3/dir2 (everyone) /ganeshavol3/dir3 (everyone) ---------------- 2.Start unexporting volume via dbus ------------ [root@dhcp42-125 exports]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:4 method return sender=:1.36 -> dest=:1.163 reply_serial=2 [root@dhcp42-125 exports]# showmount -e Export list for dhcp42-125.lab.eng.blr.redhat.com: /ganeshavol3 (everyone) /ganeshavol3/dir1 (everyone) /ganeshavol3/dir3 (everyone) [root@dhcp42-125 exports]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:3 method return sender=:1.36 -> dest=:1.166 reply_serial=2 [root@dhcp42-125 exports]# showmount -e Export list for dhcp42-125.lab.eng.blr.redhat.com: /ganeshavol3 (everyone) /ganeshavol3/dir3 (everyone) [root@dhcp42-125 exports]# dbus-send --print-reply --system --dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:5 method return sender=:1.36 -> dest=:1.167 reply_serial=2 [root@dhcp42-125 exports]# showmount -e Export list for dhcp42-125.lab.eng.blr.redhat.com: /ganeshavol3 (everyone) ----------- Moving this bug to verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:2779