+++ This bug was initially created as a clone of Bug #1365900 +++ Description of problem: Volume gets mounted with v4 even when it is unexported. Version-Release number of selected component (if applicable): [root@dhcp43-133 ~]# rpm -qa|grep glusterfs glusterfs-libs-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-fuse-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-api-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-cli-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-ganesha-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-client-xlators-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-server-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 glusterfs-geo-replication-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 [root@dhcp43-133 ~]# rpm -qa|grep ganesha nfs-ganesha-gluster-2.4-0.dev.26.el7rhgs.x86_64 nfs-ganesha-2.4-0.dev.26.el7rhgs.x86_64 glusterfs-ganesha-3.8.1-0.4.git56fcf39.el7rhgs.x86_64 How reproducible: Always Steps to Reproduce: 1.Create a ganesha cluster and create a volume. 2.Enable ganesha on the volume and mount it on a client. [root@dhcp43-133 ~]# gluster vol set testvolume ganesha.enable on volume set: success 10.70.40.192:/testvolume on /mnt/nfs1 type nfs4 (rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.70.46.206,local_lock=none,addr=10.70.40.192) 3.Mount is successful as expected. 4.Unmount the volume and unexport it. [root@dhcp43-133 ~]# gluster vol set testvolume ganesha.enable off volume set: success [root@dhcp43-133 ~]# showmount -e localhost Export list for localhost: 5.Try mounting the volume back with v4 and observe that it gets mounted and doesn't give any error [root@dhcp46-206 ~]# mount -t nfs -o vers=4 10.70.40.192:/testvolume /mnt/nfs1 [root@dhcp46-206 ~]# 10.70.40.192://testvolume on /mnt/nfs1 type nfs4 (rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.70.46.206,local_lock=none,addr=10.70.40.192) >> Trying the same with v3 mount gives access denied error as below: [root@dhcp46-206 ~]# mount -t nfs -o vers=3 10.70.40.192:/testvolume /mnt/nfs2 mount.nfs: access denied by server while mounting 10.70.40.192:/testvolume 6. Creating dir or files inside the mount point gives "read-only filesystem" [root@dhcp46-206 ~]# cd /mnt/nfs1 [root@dhcp46-206 nfs1]# mkdir dir mkdir: cannot create directory ‘dir’: Read-only file system [root@dhcp46-206 nfs1]# touch file touch: cannot touch ‘file’: Read-only file system Actual results: Volume gets mounted with ganesha v4 mount even when it is unexported. Expected results: Mounting an unexported volume with v4 ganesha mount should give proper error message and should not get mounted. Additional info: Will attach the packet trace --- Additional comment from Jiffin on 2016-08-11 06:36:32 EDT --- The issue can be easily reproduced in rpm(based on dev26) provided to qa, but on the latest ganesha dev branch(dev28-a) it is not seen. So I am putting a need info Shashank for the same to try again on next provided ganesha build. But there is another issue, I can consistently receive following error during unexport from dbus command :(even though entry is unexported properly) Error org.freedesktop.DBus.Error.NoReply: Message recipient disconnected from message bus without replying. --- Additional comment from Shashank Raj on 2016-08-23 08:33:12 EDT --- Since with the latest build (3.8.2), unexport of ganesha volume is not working properly and it is being tracked under seperate bug (https://bugzilla.redhat.com/show_bug.cgi?id=1369398). Will verify this once, the unexport issue gets resolved. Clearing needinfo as of now. --- Additional comment from Niels de Vos on 2016-09-12 01:39:25 EDT --- All 3.8.x bugs are now reported against version 3.8 (without .x). For more information, see http://www.gluster.org/pipermail/gluster-devel/2016-September/050859.html
Are you able to list the contents of mount point post unexporting the volume?
Steps to reproduce: 1. Mount the volume on client with v4 and do some IO's: root@Client2 ~]# mount -t nfs -o vers=4 10.70.40.209:/testvolume/ /mnt/nfs1 [root@Client2 ~]# df Filesystem 1K-blocks Used Available Use% 10.70.40.209:/testvolume 125107200 236544 124870656 1% /mnt/nfs1 [root@Client2 nfs1]# ls [root@Client2 nfs1]# for i in {1..10}; do mkdir /mnt/nfs1/d$i; for j in {1..10}; do mkdir /mnt/nfs1/d$i/e$j; for k in {1..10}; do touch /mnt/nfs1/d$i/e$j/f$k; done done done [root@Client2 nfs1]# ls d1 d10 d2 d3 d4 d5 d6 d7 d8 d9 2. unmount the volume and unexport it. [root@Client2 ~]# umount /mnt/nfs1 [root@dhcp43-110 ~]# gluster vol set testvolume ganesha.enable off volume set: success [root@dhcp43-110 exports]# pwd /var/run/gluster/shared_storage/nfs-ganesha/exports [root@dhcp43-110 exports]# ls [root@dhcp43-110 exports]# showmount -e localhost Export list for localhost: 3. Mount the same volume again even when it is unexported and observe that it is successful: [root@Client2 ~]# mount -t nfs -o vers=4 10.70.40.209:/testvolume/ /mnt/nfs1 [root@Client2 ~]# 10.70.40.209://testvolume on /mnt/nfs1 type nfs4 (rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.70.42.94,local_lock=none,addr=10.70.40.209) 4. Try listing the contents on the mount point and observe that it doesn't list the contents but doesn't give any error as well: [root@Client2 ~]# cd /mnt/nfs1 [root@Client2 nfs1]# ls [root@Client2 nfs1]# ls -a . [root@Client2 nfs1]# ls -ltr total 0 5. Creating a file/dir on mount point fails with read-only file system: [root@Client2 nfs1]# mkdir new mkdir: cannot create directory ‘new’: Read-only file system [root@Client2 nfs1]# touch file touch: cannot touch ‘file’: Read-only file system
Thanks Shashank. I think its the result of the pseudo exports support in NFSv4. CCin Frank to confirm that. As it doesn't list any contents I think there is no security breach. Need to check if we can avoid even mount as well. Moving this bug to 3.2.0-beyond for now
Hmm, missed this one before. I think it's a problem with cleanup of the PseudoFS, The directory SHOULD have been removed from the PseudoFS on unexport, but I myself have never done unexport testing...
Upstream fix: https://review.gerrithub.io/347972
Tried following : 1.Create a ganesha cluster and create a volume. 2.Enable ganesha on the volume and mount it on a client. 3.Mount is successful as expected. 4.Unmount the volume and unexport it. 5.Try mounting the volume back with v4 : Result: Volume doesn't get mounted and gives following error. mount -t nfs -o vers=4 10.70.44.149:/testvol /mnt/nfs mount.nfs: mounting 10.70.44.149:/testvol failed, reason given by server: No such file or directory With version 3 it gives following error: mount -t nfs -o vers=3 10.70.44.149:/testvol /mnt/nfs mount.nfs: access denied by server while mounting 10.70.44.149:/testvol Marking the BZ verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:2779