Bug 1754483 - Peer wrongly shown as Connected.
Summary: Peer wrongly shown as Connected.
Alias: None
Product: GlusterFS
Classification: Community
Component: glusterd
Version: 5
Hardware: other
OS: Linux
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
Depends On:
TreeView+ depends on / blocked
Reported: 2019-09-23 11:51 UTC by Pushkar
Modified: 2020-12-02 05:51 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2020-01-06 11:15:11 UTC
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:

Attachments (Terms of Use)

Description Pushkar 2019-09-23 11:51:30 UTC
Description of problem:
We use glusterfs to sync configuration on active/standby servers (A/B).
So, we have 2 virtual machines running debian with glusterfs setup.
We want to support upgrade/revert for the vms.
Upgrade will involve shutting down vm, detaching existing vmdk disk image, loading another vmdk disk and starting up the vm.
Revert will involve shutting down the vm and simply reattaching the earlier used vmdk disk image.
Chnaging the vmdk disk image results in peer connected(rejected) which we are able to handle.
The issue is with revert when we bring back older vmdk image.

Version-Release number of selected component (if applicable):

How reproducible:
Reproducible on vmware, virtualbox.
Reproducible on glusterfs-3.8.8 and 5.3

Steps to Reproduce:
1. Setup replicated glusterfs on 2 virtual machines - A and B (vmware/virtualbox) using vmdk as disk format.
2. Do a peer probe and observe that they form a trusted pool and show as connected.
3. Turn off virtual machine B.
4. Remove the vmdk disk currently attached to B and add a new fresh vmdk image. Proceed to setup glusterfs on the newer setup.
5. On A, we now see B as connected(rejected).
6. On A, gluster detach B.
7. On A, gluster peer probe B, to again form a trusted pool.
8. Shutdown B, remove the curently attached vmdk disk image and reattach the older vmdk diskimage and start B.
9. Once B is up, A still shows B as in cluster and connected.
10. B correctly shows A as disconnected.
The IPs used by the vms do not change during the procedure.

Actual results:
A shows B as connected at the end of all the steps.

Expected results:
A should show B as disconnected/rejected at the end of all the steps.

Additional info:

On A (after the final step):

pushkar@pushkar-VirtualBox1:~$ sudo gluster pool list
UUID					Hostname    	State
95455fbb-b26b-4071-b9a5-4f8f98decf12	Connected 
c1c2be16-6979-47e2-9560-0d02b19a3eef	localhost   	Connected 

pushkar@pushkar-VirtualBox1:~$ sudo gluster peer status
Number of Peers: 1

Uuid: 95455fbb-b26b-4071-b9a5-4f8f98decf12
State: Peer in Cluster (Connected)

On B (after the final step):

pushkar@pushkar-VirtualBox2:~$ sudo gluster pool list
UUID					Hostname    	State
95455fbb-b26b-4071-b9a5-4f8f98decf12	Disconnected 
c1c2be16-6979-47e2-9560-0d02b19a3eef 	Disconnected 
536c921f-236d-4d28-a6b9-cd250c49b1f3	localhost   	Connected 

pushkar@pushkar-VirtualBox2:~$ sudo gluster peer status
Number of Peers: 2

Uuid: 95455fbb-b26b-4071-b9a5-4f8f98decf12
State: Peer in Cluster (Disconnected)

Uuid: c1c2be16-6979-47e2-9560-0d02b19a3eef
State: Peer in Cluster (Disconnected)


I am aware that replacing the vmdk disk images results in A and B going out of sync of the knowledge they have of their peers.
But should not A show B as not connected/rejected?

Comment 1 Sanju 2019-12-05 10:18:57 UTC
I think it is a firewall issue, can you please verify?

Comment 2 Sanju 2020-01-06 11:15:11 UTC
Closing this bug. If you hit this issue again, please feel free to re-open.

Note You need to log in before you can comment on or make changes to this bug.