Bug 1356426

Summary:	Upgrade from 3.7.8 to 3.7.13 doesn't regenerate the volfiles
Product:	[Community] GlusterFS	Reporter:	Kotresh HR <khiremat>
Component:	glusterd	Assignee:	Kotresh HR <khiremat>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	3.7.13	CC:	amukherj, bsrirama, bugs, kramdoss, rcyriac, sanandpa, sankarshan
Target Milestone:	---	Keywords:	ZStream
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	glusterfs-3.7.14	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1355628
Clones:	1356439 (view as bug list)		Environment:
Last Closed:	2016-08-02 07:24:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1353470, 1355628
Bug Blocks:	1356439

Description Kotresh HR 2016-07-14 06:10:42 UTC

+++ This bug was initially created as a clone of Bug #1355628 +++

+++ This bug was initially created as a clone of Bug #1353470 +++

Description of problem:

On CentOS6 base OS, post upgrade of gluster from 3.7.8 to 3.7.13, volfiles are not recreated. During upgrade, glusterd is brought up with "--xlator-option *.upgrade=on -N" parameters to ensure the volfiles are regenerated w.r.t latest bits but that fails as glusterd init () fails from glusterd_check_gsync_present () with a error log saying "0-glusterd: geo-replication module not working as desired".

Version-Release number of selected component (if applicable):


How reproducible:
Always

Additional info:

As a workaround, post 'yum update' following needs to be done.
1. grep -irns "geo-replication module not working as desired" /var/log/glusterfs/etc-glusterfs-glusterd.vol.log | wc -l

If the output is non-zero, then go to step 2

2. Check if glusterd instance is running or not by 'ps aux | grep glusterd', if it is, then stop the glusterd service.

3. glusterd --xlator-option *.upgrade=on -N

and then proceed ahead with the rest of the steps as per the upgrade section from installation guide.

Comment 1 Vijay Bellur 2016-07-14 06:12:37 UTC

REVIEW: http://review.gluster.org/14915 (glusterd: Fix gsyncd upgrade issue) posted (#1) for review on release-3.7 by Kotresh HR (khiremat)

Comment 2 Vijay Bellur 2016-07-15 13:45:27 UTC

COMMIT: http://review.gluster.org/14915 committed in release-3.7 by Jeff Darcy (jdarcy) 
------
commit ce1905c58fd604f1ef00a55609a832d1cf658c7e
Author: Kotresh HR <khiremat>
Date:   Tue Jul 12 00:39:31 2016 +0530

    glusterd: Fix gsyncd upgrade issue
    
    Backport of http://review.gluster.org/14898/
    
    Problem:
        gluster upgrade is not generating new volfiles
    
    Cause:
    During upgrade, "glusterd --xlator-option *.upgrade=on -N"
    is run to generate new volfiles. It is run post 'glusterfs'
    rpm installation. The above command fails during upgrade
    if geo-replication is installed. This is because on
    glusterd start 'gsyncd' binary is called to configure
    geo-replication related stuff. Since 'glusterfs' rpm is
    installed prior to 'geo-rep' rpm, the 'gsyncd' binary
    used to glusterd upgrade command is of old version and
    hence it fails before generating new volfiles.
    
    Solution:
    Don't call geo-replication configure during upgrade/downgrade.
    Geo-replication configuration happens during start of glusterd
    after upgrade.
    
    Change-Id: Id58ea44ead9f69982f86fb68dc5b9ee3f6cd11a1
    BUG: 1356426
    Signed-off-by: Kotresh HR <khiremat>
    (cherry picked from commit 1b998788ece8c8b52657e8b9aae65d3279690c5b)
    Reviewed-on: http://review.gluster.org/14915
    CentOS-regression: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Jeff Darcy <jdarcy>

Comment 3 Kaushal 2016-08-02 07:24:47 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.14, please open a new bug report.

glusterfs-3.7.14 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-devel/2016-August/050319.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user