Description of problem: We use glusterfs to sync configuration on active/standby servers (A/B). So, we have 2 virtual machines running debian with glusterfs setup. We want to support upgrade/revert for the vms. Upgrade will involve shutting down vm, detaching existing vmdk disk image, loading another vmdk disk and starting up the vm. Revert will involve shutting down the vm and simply reattaching the earlier used vmdk disk image. Chnaging the vmdk disk image results in peer connected(rejected) which we are able to handle. The issue is with revert when we bring back older vmdk image. Version-Release number of selected component (if applicable): 5.3 How reproducible: Always. Reproducible on vmware, virtualbox. Reproducible on glusterfs-3.8.8 and 5.3 Steps to Reproduce: 1. Setup replicated glusterfs on 2 virtual machines - A and B (vmware/virtualbox) using vmdk as disk format. 2. Do a peer probe and observe that they form a trusted pool and show as connected. 3. Turn off virtual machine B. 4. Remove the vmdk disk currently attached to B and add a new fresh vmdk image. Proceed to setup glusterfs on the newer setup. 5. On A, we now see B as connected(rejected). 6. On A, gluster detach B. 7. On A, gluster peer probe B, to again form a trusted pool. 8. Shutdown B, remove the curently attached vmdk disk image and reattach the older vmdk diskimage and start B. 9. Once B is up, A still shows B as in cluster and connected. 10. B correctly shows A as disconnected. The IPs used by the vms do not change during the procedure. Actual results: A shows B as connected at the end of all the steps. Expected results: A should show B as disconnected/rejected at the end of all the steps. Additional info: =================================================== On A (after the final step): pushkar@pushkar-VirtualBox1:~$ sudo gluster pool list UUID Hostname State 95455fbb-b26b-4071-b9a5-4f8f98decf12 10.70.52.128 Connected c1c2be16-6979-47e2-9560-0d02b19a3eef localhost Connected pushkar@pushkar-VirtualBox1:~$ sudo gluster peer status Number of Peers: 1 Hostname: 10.70.52.128 Uuid: 95455fbb-b26b-4071-b9a5-4f8f98decf12 State: Peer in Cluster (Connected) =================================================== On B (after the final step): pushkar@pushkar-VirtualBox2:~$ sudo gluster pool list UUID Hostname State 95455fbb-b26b-4071-b9a5-4f8f98decf12 10.70.52.128 Disconnected c1c2be16-6979-47e2-9560-0d02b19a3eef 10.70.52.86 Disconnected 536c921f-236d-4d28-a6b9-cd250c49b1f3 localhost Connected pushkar@pushkar-VirtualBox2:~$ sudo gluster peer status Number of Peers: 2 Hostname: 10.70.52.128 Uuid: 95455fbb-b26b-4071-b9a5-4f8f98decf12 State: Peer in Cluster (Disconnected) Hostname: 10.70.52.86 Uuid: c1c2be16-6979-47e2-9560-0d02b19a3eef State: Peer in Cluster (Disconnected) =================================================== I am aware that replacing the vmdk disk images results in A and B going out of sync of the knowledge they have of their peers. But should not A show B as not connected/rejected?
I think it is a firewall issue, can you please verify?
Closing this bug. If you hit this issue again, please feel free to re-open.