Bug 1356439

Summary:	Upgrade from 3.7.8 to 3.8.1 doesn't regenerate the volfiles
Product:	[Community] GlusterFS	Reporter:	Kotresh HR <khiremat>
Component:	glusterd	Assignee:	Kotresh HR <khiremat>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	3.8.1	CC:	amukherj, bsrirama, bugs, kramdoss, rcyriac, sanandpa, sankarshan
Target Milestone:	---	Keywords:	ZStream
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	glusterfs-3.8.2	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1356426	Environment:
Last Closed:	2016-08-12 09:47:17 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1353470, 1355628, 1356426
Bug Blocks:

Description Kotresh HR 2016-07-14 06:49:22 UTC

+++ This bug was initially created as a clone of Bug #1356426 +++

+++ This bug was initially created as a clone of Bug #1355628 +++

+++ This bug was initially created as a clone of Bug #1353470 +++

Description of problem:

On CentOS6 base OS, post upgrade of gluster from 3.7.8 to 3.7.13, volfiles are not recreated. During upgrade, glusterd is brought up with "--xlator-option *.upgrade=on -N" parameters to ensure the volfiles are regenerated w.r.t latest bits but that fails as glusterd init () fails from glusterd_check_gsync_present () with a error log saying "0-glusterd: geo-replication module not working as desired".

Version-Release number of selected component (if applicable):


How reproducible:
Always

Additional info:

As a workaround, post 'yum update' following needs to be done.
1. grep -irns "geo-replication module not working as desired" /var/log/glusterfs/etc-glusterfs-glusterd.vol.log | wc -l

If the output is non-zero, then go to step 2

2. Check if glusterd instance is running or not by 'ps aux | grep glusterd', if it is, then stop the glusterd service.

3. glusterd --xlator-option *.upgrade=on -N

and then proceed ahead with the rest of the steps as per the upgrade section from installation guide.

--- Additional comment from Vijay Bellur on 2016-07-14 02:12:37 EDT ---

REVIEW: http://review.gluster.org/14915 (glusterd: Fix gsyncd upgrade issue) posted (#1) for review on release-3.7 by Kotresh HR (khiremat)

Comment 1 Vijay Bellur 2016-07-14 06:51:04 UTC

REVIEW: http://review.gluster.org/14916 (glusterd: Fix gsyncd upgrade issue) posted (#1) for review on release-3.8 by Kotresh HR (khiremat)

Comment 2 Vijay Bellur 2016-07-15 16:47:48 UTC

COMMIT: http://review.gluster.org/14916 committed in release-3.8 by Atin Mukherjee (amukherj) 
------
commit ddcfe83b06b6498710c2d9d36027a8ad305b686e
Author: Kotresh HR <khiremat>
Date:   Tue Jul 12 00:39:31 2016 +0530

    glusterd: Fix gsyncd upgrade issue
    
    Backport of http://review.gluster.org/14898/
    
    Problem:
        gluster upgrade is not generating new volfiles
    
    Cause:
    During upgrade, "glusterd --xlator-option *.upgrade=on -N"
    is run to generate new volfiles. It is run post 'glusterfs'
    rpm installation. The above command fails during upgrade
    if geo-replication is installed. This is because on
    glusterd start 'gsyncd' binary is called to configure
    geo-replication related stuff. Since 'glusterfs' rpm is
    installed prior to 'geo-rep' rpm, the 'gsyncd' binary
    used to glusterd upgrade command is of old version and
    hence it fails before generating new volfiles.
    
    Solution:
    Don't call geo-replication configure during upgrade/downgrade.
    Geo-replication configuration happens during start of glusterd
    after upgrade.
    
    Change-Id: Id58ea44ead9f69982f86fb68dc5b9ee3f6cd11a1
    BUG: 1356439
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: http://review.gluster.org/14898
    (cherry picked from commit 1b998788ece8c8b52657e8b9aae65d3279690c5b)
    Reviewed-on: http://review.gluster.org/14916
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Atin Mukherjee <amukherj>

Comment 3 Niels de Vos 2016-08-12 09:47:17 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.2, please open a new bug report.

glusterfs-3.8.2 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://www.gluster.org/pipermail/announce/2016-August/000058.html
[2] https://www.gluster.org/pipermail/gluster-users/