+++ This bug was initially created as a clone of Bug #1551186 +++ Description of problem: While exporting multiple volumes,duplicate volume entries are been observed in /etc/ganesha/ganesha.conf which leads to failure while unexporting volumes Version-Release number of selected component (if applicable): 3.10 How reproducible: 3/3 Steps to Reproduce: 1.Create 4 node ganesha cluster 2.Create and start 25 volumes 3.Export all volumes via ganesha for i in $(seq 1 25) do gluster v set vol$i ganesha.enable on sleep 10 done 4.Unexport the ganesha volumes for i in $(seq 1 25) do gluster v set vol$i ganesha.enable off sleep 10 done Actual results: In /etc/ganesha/ganesha.conf ,duplicate volume export entries are observed causing volume unexport failed. ======== #/etc/ganesha/ganesha.conf EXPORT_DEFAULTS { # To reflect nfsnobody Anonymous_uid = 65534; Anonymous_gid = 65534; } %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol1.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol2.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol3.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol4.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol5.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol6.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol7.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol8.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol9.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol10.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol11.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol12.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol13.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol14.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol15.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol15.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol16.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol17.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol18.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol18.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol19.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol20.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol20.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol21.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol21.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol22.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol23.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol23.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol24.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol25.conf" %include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol25.conf" [root@dhcp37-218 exports]# ls export.vol10.conf export.vol14.conf export.vol18.conf export.vol21.conf export.vol25.conf export.vol5.conf export.vol9.conf export.vol11.conf export.vol15.conf export.vol19.conf export.vol22.conf export.vol2.conf export.vol6.conf export.vol12.conf export.vol16.conf export.vol1.conf export.vol23.conf export.vol3.conf export.vol7.conf export.vol13.conf export.vol17.conf export.vol20.conf export.vol24.conf export.vol4.conf export.vol8.conf # showmount -e Export list for dhcp37-218.lab.eng.blr.redhat.com: /vol1 (everyone) /vol2 (everyone) /vol3 (everyone) /vol4 (everyone) /vol5 (everyone) /vol6 (everyone) /vol7 (everyone) /vol8 (everyone) /vol9 (everyone) /vol10 (everyone) /vol11 (everyone) /vol12 (everyone) /vol13 (everyone) /vol14 (everyone) /vol15 (everyone) /vol16 (everyone) /vol17 (everyone) /vol18 (everyone) /vol19 (everyone) /vol20 (everyone) /vol21 (everyone) /vol22 (everyone) /vol23 (everyone) /vol24 (everyone) /vol25 (everyone) Volume unexport- # sh ganesha_export.sh volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: success volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details volume set: success volume set: success volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details volume set: success volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details volume set: success volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details volume set: success volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details ======================= Expected results: Volume export/unexport should work fine Additional info: --- Additional comment from Jiffin on 2018-03-13 04:26:11 EDT --- There is possibility of having multiple entries get created in ganesha.conf because create-ganesha-export.sh during volume set ganesha.enable command and can be executed from multiple nodes at the same time. Even though script checks for duplicate entry, it cannot trusted properly. So it is better to execute the script from only one node, so above scenarios can be avoided. Second unexport failure due to duplicate entries I still need to debug that
REVIEW: https://review.gluster.org/19716 (glusterd/ganesha : create/remove export file only from the node which performs ganesha.enable) posted (#1) for review on release-3.10 by jiffin tony Thottan
COMMIT: https://review.gluster.org/19716 committed in release-3.10 by "Shyamsundar Ranganathan" <srangana> with a commit message- glusterd/ganesha : create/remove export file only from the node which performs ganesha.enable As part of volume set ganesha.enable on the ganesha export configuration file will be created/removed using "create-export-ganesha.sh". This performed from the nodes which are part of ganesha cluster. But it is not need since the file is saved in shared storage and consumed by the nodes in the ganesha cluster. Change-Id: I2583899972b47d451a90711940293004a3af4690 BUG: 1555195 Signed-off-by: Jiffin Tony Thottan <jthottan>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.12, please open a new bug report. glusterfs-3.10.12 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2018-April/000095.html [2] https://www.gluster.org/pipermail/gluster-users/