Bug 1351211

Summary: [RFE] Add peer detach force when detaching host that is non-operational
Product: [oVirt] ovirt-engine Reporter: RamaKasturi <knarra>
Component: BLL.GlusterAssignee: Gobinda Das <godas>
Status: CLOSED WORKSFORME QA Contact: SATHEESARAN <sasundar>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.6.7CC: bugs, knarra, sabose, trichard
Target Milestone: ovirt-4.1.5Keywords: FutureFeature
Target Release: ---Flags: sasundar: ovirt-4.1?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
When removing a Gluster host after moving it to maintenance mode, the host is not removed from the peer list of other hosts. To work around this issue, do not stop the Gluster services while moving the host to maintenance mode if the host is going to be removed from cluster.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-18 06:38:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Gluster RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1411323    

Description RamaKasturi 2016-06-29 13:23:10 UTC
Description of problem:
Peer detach works only when glusterd service is running on the node being detached. In cases where the services are not running on old peer, peer detach from UI does not work.

Version-Release number of selected component (if applicable):
rhevm-3.6.7.5-0.1.el6.noarch

How reproducible:
Always

Steps to Reproduce:
1. Install HC setup
2. Now poweroff one of the node and wait for it to go to non-responsive state.
3. Move the node to maintenance and click on remove.

Actual results:
Node gets removed from UI but in cli it is still listed in the peer status as disconnected.

Expected results:
peer detach of the host should happen.

Additional info:

Comment 1 Sahina Bose 2016-07-26 10:59:49 UTC
We have now introduced a way to optionally stop glusterd services while moving a host to maintenance. When you wish to detach the host from cluster, the flow is to move host to maintenance without stopping glusterd services, and then remove from cluster.

Comment 2 Sahina Bose 2016-07-26 12:24:52 UTC
Re-opening as I misread the bug

Comment 3 Sahina Bose 2016-12-22 05:51:10 UTC
The issue of the host being removed remaining in peer list only happens when gluster services are stopped while moving host to maintenance. Considering this, moving this out of 4.1

Comment 4 RamaKasturi 2017-03-23 13:09:54 UTC
This bug is important during replace node with different FQDN flow where user will have to replace a node which is completely down. 

For example, take a case where the user wants to replace a node which is completely dead. In this case engine will not be able to reach the node which is down and when a node is removed from UI it only does 'gluster peer detach' which removes it from the UI but it will still be present in the 'gluster peer status'.

 To remove this from gluster peer status user has to perform 'gluster peer detach <host> force'

Comment 5 Gobinda Das 2017-07-14 07:30:31 UTC
Tested locally and it works as expected.
Below steps i followed:
 Scenario-1

1) Created two host from UI.
2) Checked "gluster peer status" in both node and found 1.
3) Stop gluster service in one node.
4) From UI moved that host to maintenance and removed it with the force checkbox enabled.
5) Host removed from UI.
6) Checked "gluster peer status" in other host where gluster running and found "Number of Peers: 0"
7) Checked /var/log/glusterfs/cmd_history.log and found "[2017-07-14 06:24:22.427552] : peer detach 10.70.42.63 force : SUCCESS"

Scenario-2

1) Created two host from UI.
2) Checked "gluster peer status" in both node and found 1.
3) Power Off one node.
4) In UI wait till host becomes unresponsive then moved that host to maintenance and removed it with the force checkbox enabled.
5) Host removed from UI.
6) Checked "gluster peer status" in other host where gluster running and found "Number of Peers: 0"

Comment 6 Sahina Bose 2017-07-14 11:27:26 UTC
Based on Comment 5, the feature requested is already implemented. Do we need further action here?

Comment 7 RamaKasturi 2017-07-14 11:36:27 UTC
Sahina,

   I would like to re-check this myself  and update the bug since this is a very old bug and i am sure i am missing something here to explain. I am trying to reproduce it and can i update the bug once i have the info ?

Thanks
kasturi.

Comment 8 RamaKasturi 2017-07-14 11:39:23 UTC
I will have need info on me till i provide the required inputs.

Comment 9 RamaKasturi 2017-07-17 12:54:16 UTC
Gobinda / Sahina,

   This bug can be closed for now and will reopen if i hit the issue again.

Thanks
kasturi

Comment 10 Sahina Bose 2017-07-18 06:38:27 UTC
Closing as per comment 5 and comment 9