Bug 1344605

Summary: [geo-rep]: Add-Brick use case: create push-pem force on existing geo-rep fails
Product: [Community] GlusterFS Reporter: Saravanakumar <sarumuga>
Component: geo-replicationAssignee: Saravanakumar <sarumuga>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.7.12CC: avishwan, bugs, csaba, rcyriac, rhinduja, storage-qa-internal
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.7.12 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1342979
: 1344607 (view as bug list) Environment:
Last Closed: 2016-06-28 12:19:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1342938, 1342979    
Bug Blocks: 1344607    

Description Saravanakumar 2016-06-10 07:27:54 UTC
Description of problem:
=======================

Known ways to add a brick from new node is to by following steps:

1. gsec create
2. create push-pem force

But with the recent validation check, the create push-pem fails complaining that the geo-rep session exists. And hence gsyncd on new node doesn't start.

If the master volume and slave volume remains same, and also if user and host remains same, then the force should not fail.

[root@dhcp37-88 ~]# gluster system:: execute gsec_create 
Common secret pub file present at /var/lib/glusterd/geo-replication/common_secret.pem.pub
[root@dhcp37-88 ~]# #gluster volume geo-replication master_nr rahul.37.52::slave_nr create push-pem
[root@dhcp37-88 ~]# gluster volume geo-replication status
 
MASTER NODE    MASTER VOL    MASTER BRICK      SLAVE USER    SLAVE                       SLAVE NODE      STATUS     CRAWL STATUS       LAST_SYNCED                  
---------------------------------------------------------------------------------------------------------------------------------------------------------
10.70.37.88    vol24         /rhs/brick1/b1    root          ssh://10.70.37.52::vol25    10.70.37.190    Active     Changelog Crawl    2016-06-05 07:31:49          
10.70.37.88    vol24         /rhs/brick2/b3    root          ssh://10.70.37.52::vol25    10.70.37.190    Active     Changelog Crawl    2016-06-05 07:31:49          
10.70.37.43    vol24         /rhs/brick1/b2    root          ssh://10.70.37.52::vol25    10.70.37.52     Passive    N/A                N/A                          
10.70.37.43    vol24         /rhs/brick2/b4    root          ssh://10.70.37.52::vol25    10.70.37.52     Passive    N/A                N/A                          
[root@dhcp37-88 ~]# 
[root@dhcp37-88 ~]# gluster volume geo-replication vol24 10.70.37.52::vol25 create push-pem force
Geo -replication session between vol24 and 10.70.37.52::vol25 is still active. Please stop the session and retry.
geo-replication command failed
[root@dhcp37-88 ~]#



Version-Release number of selected component (if applicable):
=============================================================


How reproducible:
=================
Always


Steps to Reproduce:
===================
1. Have existing geo-rep session
2. Add new node on Master cluster
3. gsec_create
4. create push-pem force with same master,slave,user and hostname

Actual results:
===============

create push-pem fails and gsync doesn't get started

--- Additional comment from Rahul Hinduja on 2016-06-06 03:43:27 EDT ---

Work around available: Use stop force before create push-pem force

* gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL
stop force
* gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL
create push-pem force

--- Additional comment from Vijay Bellur on 2016-06-10 03:20:22 EDT ---

COMMIT: http://review.gluster.org/14653 committed in master by Aravinda VK (avishwan) 
------
commit c62493efadbcf5085bbd65a409eed9391301c154
Author: Saravanakumar Arumugam <sarumuga>
Date:   Mon Jun 6 14:44:35 2016 +0530

    glusterd/geo-rep: Avoid started status check if same host
    
    After carrying out add-brick, session creation is carried out
    again, to involve new brick in the session. This needs to be done,
    even if the session is in Started state.
    
    While involving slave uuid as part of a session, User is warned
    if session is in Started state. This check needs to be avoided
    if it is same slave host and session creation needs to be proceeded.
    
    Change-Id: Ic73edd5bd9e3ee55da96f5aceec0bafa14d3f3dd
    BUG: 1342979
    Signed-off-by: Saravanakumar Arumugam <sarumuga>
    Reviewed-on: http://review.gluster.org/14653
    CentOS-regression: Gluster Build System <jenkins.com>
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Aravinda VK <avishwan>

Comment 1 Vijay Bellur 2016-06-13 10:41:14 UTC
REVIEW: http://review.gluster.org/14710 (glusterd/geo-rep: Avoid started status check if same host) posted (#1) for review on release-3.7 by Saravanakumar Arumugam (sarumuga)

Comment 2 Vijay Bellur 2016-06-14 03:18:57 UTC
REVIEW: http://review.gluster.org/14710 (glusterd/geo-rep: Avoid started status check if same host) posted (#2) for review on release-3.7 by Aravinda VK (avishwan)

Comment 3 Vijay Bellur 2016-06-14 06:23:18 UTC
COMMIT: http://review.gluster.org/14710 committed in release-3.7 by Aravinda VK (avishwan) 
------
commit 0eddfc43be1939f4fe892cd09c7356091b243b3a
Author: Saravanakumar Arumugam <sarumuga>
Date:   Mon Jun 6 14:44:35 2016 +0530

    glusterd/geo-rep: Avoid started status check if same host
    
    After carrying out add-brick, session creation is carried out
    again, to involve new brick in the session. This needs to be done,
    even if the session is in Started state.
    
    While involving slave uuid as part of a session, User is warned
    if session is in Started state. This check needs to be avoided
    if it is same slave host and session creation needs to be proceeded.
    
    Change-Id: Ic73edd5bd9e3ee55da96f5aceec0bafa14d3f3dd
    BUG: 1344605
    Signed-off-by: Saravanakumar Arumugam <sarumuga>
    Reviewed-on: http://review.gluster.org/14653
    CentOS-regression: Gluster Build System <jenkins.com>
    Smoke: Gluster Build System <jenkins.com>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Reviewed-by: Aravinda VK <avishwan>
    (cherry picked from commit c62493efadbcf5085bbd65a409eed9391301c154)
    Reviewed-on: http://review.gluster.org/14710
    Tested-by: Aravinda VK <avishwan>

Comment 4 Kaushal 2016-06-28 12:19:47 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.12, please open a new bug report.

glusterfs-3.7.12 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-devel/2016-June/049918.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user