A bug fixes for above feature +++ This bug was initially created as a clone of Bug #1355956 +++ Description of problem: In the present design, we maintain a copy of the below database (required for NFS-Ganesha setup) on all the nodes of the Gluster Storage pool - * /etc/ganesha/ganesha-ha.conf (with the details about the nodes forming NFS-Ganesha cluster and their VIPs to be configured) * /etc/ganesha/ganesha.conf (contains configuration parameters and the volumes list to be exported, used by NFS-Ganesha process while starting) * /etc/ganesha/exports/ganesha.$vol.conf (config file of a volume created when a volume is being exported by NFS-Ganesha) * /etc/ganesha/exports/.export_added (each volume should have unique ExportID. This file is to keep track of the last ExportID used) Outstanding Issues: Since each node shall have a local copy, we have few issues with respect to synchronizing them across the nodes. Few of them are outlined below: * When a node reboots, depending on the order the services ‘pacemaker’ and ‘glusterd’ starts, we may run into a case, where in glusterd couldn’t sync the export files on this node. * There could be case where in ‘.export_added’ could go out of sync across the nodes thus resulting in different ExportIDs for the same volume exported across different nfs-ganesha heads. This shall result in NFS mounts throwing “Stale File handle error” post failover * When a node is down while performing “refresh-config”, the export config of that volume doesn’t get synced when it comes back up. * As one of the pre-requisites, ganesha-ha.conf has to be copied to all the nodes in the gluster cluster before enabling nfs-ganesha (setup). * When any changes are made to main “/etc/ganesha.conf” file, it has to be manually copied to all the nodes in the cluster. * When “refresh-config” is performed, the script syncs up the volume config only across the NFS-Ganesha cluster but not to all the nodes in the Gluster storage pool Version-Release number of selected component (if applicable): mainline Additional info: The existing users shall be impacted while doing upgrade. We may need to provide scripts for the smooth upgrade to move the older config data to shared_storage and also few of the setup pre-requisites need to be removed/modified accordingly. --- Additional comment from Vijay Bellur on 2016-07-13 05:55:25 EDT --- REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move ganesha-ha.conf and ganesha.conf to shared storage) posted (#1) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-07-13 05:55:28 EDT --- REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy ganesha export configuration files during reboot") posted (#1) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-07-13 05:55:31 EDT --- REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export configuration file in shared storage) posted (#1) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-07-13 05:55:35 EDT --- REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying ganesha-ha.sh for share storage related changes) posted (#1) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-07-15 05:24:44 EDT --- REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export configuration file in shared storage) posted (#2) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-07-15 05:24:47 EDT --- REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying ganesha-ha.sh for share storage related changes) posted (#2) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-07-15 05:24:50 EDT --- REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move ganesha-ha.conf and ganesha.conf to shared storage) posted (#2) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-07-15 05:24:54 EDT --- REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy ganesha export configuration files during reboot") posted (#2) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-08-08 05:25:14 EDT --- REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export configuration file in shared storage) posted (#3) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-08-08 05:25:17 EDT --- REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying ganesha-ha.sh for share storage related changes) posted (#3) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-08-08 05:25:22 EDT --- REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move ganesha-ha.conf and ganesha.conf to shared storage) posted (#3) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Vijay Bellur on 2016-08-08 05:25:26 EDT --- REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy ganesha export configuration files during reboot") posted (#3) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Worker Ant on 2016-08-25 04:37:43 EDT --- REVIEW: http://review.gluster.org/14908 (glusterd/ganesha : create export configuration file in shared storage) posted (#4) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Worker Ant on 2016-08-25 04:37:47 EDT --- REVIEW: http://review.gluster.org/14909 (ganesha/scripts : Modifying ganesha-ha.sh for share storage related changes) posted (#4) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Worker Ant on 2016-08-25 04:37:51 EDT --- REVIEW: http://review.gluster.org/14906 (glusterd/ganesha : Move ganesha-ha.conf and ganesha.conf to shared storage) posted (#4) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Worker Ant on 2016-08-25 04:37:55 EDT --- REVIEW: http://review.gluster.org/14907 (Revert "glusterd-ganesha : copy ganesha export configuration files during reboot") posted (#4) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Worker Ant on 2016-08-25 09:56:01 EDT --- COMMIT: http://review.gluster.org/14906 committed in master by Kaleb KEITHLEY (kkeithle) ------ commit 76726da0e86077a8f3a59c02a47fcf2e3994218f Author: Jiffin Tony Thottan <jthottan> Date: Tue Jul 12 15:44:23 2016 +0530 glusterd/ganesha : Move ganesha-ha.conf and ganesha.conf to shared storage Currently all the ganesha related configuration files(ganesha.conf, ganesha-ha.conf, export files, etc) is stored locally at /etc/ganesha on a every node in ganesha cluster. Usually we end up in two issues by doing so : * difficult in modifiying ganesha related conf file * diffciult to maintain consistency of conf file across ganesha cluster To tackle this, we plan to move all the ganesha configuration to shared storage. As a first step in this patch ganesha.conf and ganesha-ha.conf move to shared storage. Here actual ganesha.conf will resides in shared stoarge and symlinks will be created in /etc/ganesha when the option "gluster nfs-ganesha enable" is executed and remove those during the "disable" part. Modified prerequisites to done before running globaloption: * enable shared storage * create nfs-ganesha folder in shared storage * create ganesha.conf and ganesha-ha.conf in it More details can be found at http://review.gluster.org/#/c/15105/ Change-Id: Ifabb6c5db50061f077a03932940190af74e2ca7f BUG: 1355956 Signed-off-by: Jiffin Tony Thottan <jthottan> Reviewed-on: http://review.gluster.org/14906 Reviewed-by: soumya k <skoduri> NetBSD-regression: NetBSD Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org> Reviewed-by: Kaleb KEITHLEY <kkeithle> CentOS-regression: Gluster Build System <jenkins.org> --- Additional comment from Worker Ant on 2016-08-26 07:46:10 EDT --- COMMIT: http://review.gluster.org/14907 committed in master by Kaleb KEITHLEY (kkeithle) ------ commit a769be2c0d4227fead7348e769bd1ae49e267971 Author: Jiffin Tony Thottan <jthottan> Date: Tue Jul 12 17:23:03 2016 +0530 Revert "glusterd-ganesha : copy ganesha export configuration files during reboot" This reverts commit f71e2fa49af185779b9f43e146effd122d4e9da0. Reason: As part of sync up node reboot this patch copies ganesha export conf file from a source node. This change is no more require if the export files are available in shared storage. Change-Id: Id9c1ae78377bbd7d5d80aa1c14f534e30feaae97 BUG: 1355956 Signed-off-by: Jiffin Tony Thottan <jthottan> Reviewed-on: http://review.gluster.org/14907 Reviewed-by: soumya k <skoduri> Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Kaleb KEITHLEY <kkeithle> --- Additional comment from Worker Ant on 2016-08-26 07:47:22 EDT --- COMMIT: http://review.gluster.org/14909 committed in master by Kaleb KEITHLEY (kkeithle) ------ commit 2f44a979132a43c1cf12ed2b9e9e94dbe1311c9e Author: Jiffin Tony Thottan <jthottan> Date: Wed Jul 13 12:09:43 2016 +0530 ganesha/scripts : Modifying ganesha-ha.sh for share storage related changes Currently the ganesha related configurations are "scp"ied for operations like add, delete, refresh-config in ganesha-ha.sh. This is no more required since all the conf files are available in shared storage and every node can directly access them from shared storage. More details can be found at http://review.gluster.org/#/c/15105/ Change-Id: Ic025eb4dc246db61d6fbe969ca60214751fbf3ba BUG: 1355956 Signed-off-by: Jiffin Tony Thottan <jthottan> Reviewed-on: http://review.gluster.org/14909 NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: soumya k <skoduri> Smoke: Gluster Build System <jenkins.org> Reviewed-by: Kaleb KEITHLEY <kkeithle> --- Additional comment from Worker Ant on 2016-08-26 07:47:55 EDT --- COMMIT: http://review.gluster.org/14908 committed in master by Kaleb KEITHLEY (kkeithle) ------ commit 4786150ebe36154ec0762a951612f66f68a7e95c Author: Jiffin Tony Thottan <jthottan> Date: Wed Jul 13 11:38:10 2016 +0530 glusterd/ganesha : create export configuration file in shared storage This is the second patch which moves export related configuration for a volume into shared storage. The main change includes in scripts create-export-ganesha.sh, dbus-send.sh and the handling of volume set command "ganesha.enable". The manipulation of EXPORT_ID move from dbus-send.sh to create-export-ganesha.sh. In volume set handling following has performed stage | commit ---------------------------------------------------------- 1.) gluster v set <volname> ganesha.enable on None | create export file | in node where cli executed, | thne export volume via dbus 2.) gluster v set <volname> ganesha.enable off unexport volume via dbus | remove export file from the | shared storage ----------------------------------------------------------- More details can be found at http://review.gluster.org/#/c/15105/ Change-Id: Ia8b0e89bc8fff24b0bc5d20a538a89212894a8e4 BUG: 1355956 Signed-off-by: Jiffin Tony Thottan <jthottan> Reviewed-on: http://review.gluster.org/14908 NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: soumya k <skoduri> Smoke: Gluster Build System <jenkins.org> Reviewed-by: Kaleb KEITHLEY <kkeithle> --- Additional comment from Worker Ant on 2016-08-30 09:11:32 EDT --- REVIEW: http://review.gluster.org/15355 (ganesha/glustrd : Correct the path for ganesha conf dir in Makefile) posted (#1) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Worker Ant on 2016-09-16 01:45:31 EDT --- REVIEW: http://review.gluster.org/15355 (ganesha/glusterd : Correct the path for ganesha conf dir in Makefile) posted (#2) for review on master by jiffin tony Thottan (jthottan) --- Additional comment from Worker Ant on 2016-09-16 10:22:54 EDT --- COMMIT: http://review.gluster.org/15355 committed in master by Atin Mukherjee (amukherj) ------ commit 7e18e16f9f62ed95acded0fb3f2a8784087f59c0 Author: Jiffin Tony Thottan <jthottan> Date: Tue Aug 30 18:36:51 2016 +0530 ganesha/glusterd : Correct the path for ganesha conf dir in Makefile The value for ganesha conf dir in Makefile.am is mistakenly entered as following : DCONFDIR=\"/$(runstatedir)/gluster/shared_storage/nfs-ganesha\" Here value for runstatedir is seems to "NULL" which results wrong path for ganesha configuration directory Change-Id: I0b7ebd8e2503de0cb79b601553c4405d0d1fd711 BUG: 1355956 Signed-off-by: Jiffin Tony Thottan <jthottan> Reviewed-on: http://review.gluster.org/15355 NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: soumya k <skoduri> Reviewed-by: Niels de Vos <ndevos> CentOS-regression: Gluster Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org>
REVIEW: http://review.gluster.org/15521 (ganesha/glusterd : Correct the path for ganesha conf dir in Makefile) posted (#1) for review on release-3.9 by jiffin tony Thottan (jthottan)
COMMIT: http://review.gluster.org/15521 committed in release-3.9 by Atin Mukherjee (amukherj) ------ commit b7bb059865c43e3098bd5682aedd0b9b716d9d23 Author: Jiffin Tony Thottan <jthottan> Date: Tue Aug 30 18:36:51 2016 +0530 ganesha/glusterd : Correct the path for ganesha conf dir in Makefile The value for ganesha conf dir in Makefile.am is mistakenly entered as following : DCONFDIR=\"/$(runstatedir)/gluster/shared_storage/nfs-ganesha\" Here value for runstatedir is seems to "NULL" which results wrong path for ganesha configuration directory Upstream reference : > Change-Id: I0b7ebd8e2503de0cb79b601553c4405d0d1fd711 > BUG: 1355956 > Signed-off-by: Jiffin Tony Thottan <jthottan> > Reviewed-on: http://review.gluster.org/15355 > NetBSD-regression: NetBSD Build System <jenkins.org> > Reviewed-by: soumya k <skoduri> > Reviewed-by: Niels de Vos <ndevos> > CentOS-regression: Gluster Build System <jenkins.org> > Smoke: Gluster Build System <jenkins.org> > (cherry picked from commit 7e18e16f9f62ed95acded0fb3f2a8784087f59c0) Change-Id: I0b7ebd8e2503de0cb79b601553c4405d0d1fd711 BUG: 1376874 Signed-off-by: Jiffin Tony Thottan <jthottan> Reviewed-on: http://review.gluster.org/15521 Reviewed-by: soumya k <skoduri> Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> Reviewed-by: Atin Mukherjee <amukherj>
glusterfs-3.9.0rc2 is released[1] and packages are available for different distributions[2] to test. [1] http://www.gluster.org/pipermail/maintainers/2016-October/001601.html [2] http://www.gluster.org/pipermail/maintainers/2016-October/001605.html and http://www.gluster.org/pipermail/maintainers/2016-October/001606.html
Gluster 3.9 GA is released http://blog.gluster.org/2016/11/announcing-gluster-3-9/