Bug 1555195 - [Ganesha] Duplicate volume export entries in ganesha.conf causing volume unexport to fail
Summary: [Ganesha] Duplicate volume export entries in ganesha.conf causing volume unex...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: 3.10
Hardware: Unspecified
OS: Unspecified
urgent
unspecified
Target Milestone: ---
Assignee: Jiffin
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1551186
TreeView+ depends on / blocked
 
Reported: 2018-03-14 06:40 UTC by Jiffin
Modified: 2018-05-07 15:05 UTC (History)
8 users (show)

Fixed In Version: glusterfs-3.10.12
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1551186
Environment:
Last Closed: 2018-05-07 15:05:04 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Jiffin 2018-03-14 06:40:50 UTC
+++ This bug was initially created as a clone of Bug #1551186 +++

Description of problem:

While exporting multiple volumes,duplicate volume entries are been observed in /etc/ganesha/ganesha.conf which leads to failure while unexporting volumes

Version-Release number of selected component (if applicable):

3.10


How reproducible:
3/3


Steps to Reproduce:

1.Create 4 node ganesha cluster
2.Create and start 25 volumes
3.Export all volumes via ganesha
for i in $(seq 1 25)
do
        gluster v set vol$i ganesha.enable on
        sleep 10
done
4.Unexport the ganesha volumes
for i in $(seq 1 25)
do
        gluster v set vol$i ganesha.enable off
        sleep 10
done

Actual results:
In /etc/ganesha/ganesha.conf ,duplicate volume export entries are observed causing volume unexport failed.

========
#/etc/ganesha/ganesha.conf


EXPORT_DEFAULTS {
	# To reflect nfsnobody
        Anonymous_uid = 65534;
        Anonymous_gid = 65534;
}
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol1.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol2.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol3.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol4.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol5.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol6.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol7.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol8.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol9.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol10.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol11.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol12.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol13.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol14.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol15.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol15.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol16.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol17.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol18.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol18.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol19.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol20.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol20.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol21.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol21.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol22.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol23.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol23.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol24.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol25.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.vol25.conf"



[root@dhcp37-218 exports]# ls
export.vol10.conf  export.vol14.conf  export.vol18.conf  export.vol21.conf  export.vol25.conf  export.vol5.conf  export.vol9.conf
export.vol11.conf  export.vol15.conf  export.vol19.conf  export.vol22.conf  export.vol2.conf   export.vol6.conf
export.vol12.conf  export.vol16.conf  export.vol1.conf   export.vol23.conf  export.vol3.conf   export.vol7.conf
export.vol13.conf  export.vol17.conf  export.vol20.conf  export.vol24.conf  export.vol4.conf   export.vol8.conf


# showmount -e
Export list for dhcp37-218.lab.eng.blr.redhat.com:
/vol1  (everyone)
/vol2  (everyone)
/vol3  (everyone)
/vol4  (everyone)
/vol5  (everyone)
/vol6  (everyone)
/vol7  (everyone)
/vol8  (everyone)
/vol9  (everyone)
/vol10 (everyone)
/vol11 (everyone)
/vol12 (everyone)
/vol13 (everyone)
/vol14 (everyone)
/vol15 (everyone)
/vol16 (everyone)
/vol17 (everyone)
/vol18 (everyone)
/vol19 (everyone)
/vol20 (everyone)
/vol21 (everyone)
/vol22 (everyone)
/vol23 (everyone)
/vol24 (everyone)
/vol25 (everyone)


Volume unexport-

# sh ganesha_export.sh 
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: success
volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
volume set: success
volume set: success
volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
volume set: success
volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
volume set: success
volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
volume set: success
volume set: failed: Staging failed on dhcp37-103.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details
Staging failed on dhcp37-136.lab.eng.blr.redhat.com. Error: Dynamic export addition/deletion failed. Please see log file for details



=======================


Expected results:

Volume export/unexport should work fine


Additional info:

--- Additional comment from Jiffin on 2018-03-13 04:26:11 EDT ---

There is possibility of having multiple entries get created in ganesha.conf because create-ganesha-export.sh during volume set ganesha.enable command and can be executed from multiple nodes at the same time. Even though script checks for duplicate entry, it cannot trusted properly. So it is better to execute the script from only one node, so above scenarios can be avoided.

Second unexport failure due to duplicate entries I still need to debug that

Comment 1 Worker Ant 2018-03-14 06:42:27 UTC
REVIEW: https://review.gluster.org/19716 (glusterd/ganesha : create/remove export file only from the node which performs ganesha.enable) posted (#1) for review on release-3.10 by jiffin tony Thottan

Comment 2 Worker Ant 2018-03-16 13:36:35 UTC
COMMIT: https://review.gluster.org/19716 committed in release-3.10 by "Shyamsundar Ranganathan" <srangana@redhat.com> with a commit message- glusterd/ganesha : create/remove export file only from the node which performs ganesha.enable

As part of volume set ganesha.enable on the ganesha export configuration file will be created/removed
using "create-export-ganesha.sh". This performed from the nodes which are part of ganesha cluster.
But it is not need since the file is saved in shared storage and consumed by the nodes in the ganesha cluster.

Change-Id: I2583899972b47d451a90711940293004a3af4690
BUG: 1555195
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>

Comment 3 Shyamsundar 2018-05-07 15:05:04 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.12, please open a new bug report.

glusterfs-3.10.12 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/announce/2018-April/000095.html
[2] https://www.gluster.org/pipermail/gluster-users/


Note You need to log in before you can comment on or make changes to this bug.