Description of problem:

We restore 1 peer from a VM Snapshot. This peer is a member of 3-replica volumes. We would like to sync the brick data from the 2 proper bricks in the replica.

When doing a "gluster volume heal <vol> full" we expect that this will replicate to the restored brick....it does only partly, all changed and new files are updated, but files/directories that were deleted (or renamed) after the restore point (but existed in the restore!) are still there.


Version-Release number of selected component (if applicable):
3.12.6

How reproducible:
Create a 3 node gluster
Create a 3 replica volume and put some data
Create a VM snapshot (eg not a gluster snapshot!) of the nodes
Change, Add, Rename, Remove some data
Restore the VM snapshot for 1 node
Run gluster volume heal <vol> full.

Actual results:
Check the data on the brick level of the restored node. You will still see removed files/dirs and for renamed files/dirs you will have both the "old" and the new name.

Expected results:
Brick data is consistent on all bricks.

Additional info:
This is part of a restore scenario where 1 gluster node is determined corrupt (in whatever area, doesn't have to be gluster) and will be restored using a VM snapshot. The VM Snapshot restore is fast, keeps all IP's, uuids (to which heketi in kubernetes and its database points!) so removing replicas, bricks, rebuilding, modifying heketi database doesn't feel like a good scenario.
If there are better ways to do this we'd be interested as well, but I'm wondering how people feel about the usage of the heal command and the expected result.