Bug 2177167

Summary: [RBD-mirror] Disabling some image feature from primary in secondary it just raise the flag but features field still have it
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Sunil Angadi <sangadi>
Component: RBD-MirrorAssignee: Ilya Dryomov <idryomov>
Status: NEW --- QA Contact: Sunil Angadi <sangadi>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 5.3CC: ceph-eng-bugs, cephqe-warriors
Target Milestone: ---   
Target Release: 6.1z2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sunil Angadi 2023-03-10 08:55:24 UTC 
Description of problem:
In snapshot based mirroring after disabling some image feature from primary,
then after resync of images in secondary image info still holds the flags of invalid feature in `features` section.

e.g 

Disabled `object-map,fast-diff` image feature from primary
[ceph: root@ceph-rbd1-sangadi-udsc8d-node1-installer /]# rbd info rep_pool_DqufdZqgib/rep_image_WEbRXwjgSo
rbd image 'rep_image_WEbRXwjgSo':
	size 1 GiB in 256 objects
	order 22 (4 MiB objects)
	snapshot_count: 1
	id: 3a3b68a80206
	block_name_prefix: rbd_data.3a3b68a80206
	format: 2
	features: layering, exclusive-lock, object-map, fast-diff, deep-flatten
	op_features: 
	flags: 
	create_timestamp: Fri Mar 10 08:06:24 2023
	access_timestamp: Fri Mar 10 08:06:24 2023
	modify_timestamp: Fri Mar 10 08:21:29 2023
	mirroring state: enabled
	mirroring mode: snapshot
	mirroring global id: 3d66e921-1c82-46cb-a9e9-164cbdac5fba
	mirroring primary: true
[ceph: root@ceph-rbd1-sangadi-udsc8d-node1-installer /]# rbd feature disable rep_pool_DqufdZqgib/rep_image_WEbRXwjgSo object-map,fast-diff
[ceph: root@ceph-rbd1-sangadi-udsc8d-node1-installer /]# 
[ceph: root@ceph-rbd1-sangadi-udsc8d-node1-installer /]# rbd info rep_pool_DqufdZqgib/rep_image_WEbRXwjgSo
rbd image 'rep_image_WEbRXwjgSo':
	size 1 GiB in 256 objects
	order 22 (4 MiB objects)
	snapshot_count: 1
	id: 3a3b68a80206
	block_name_prefix: rbd_data.3a3b68a80206
	format: 2
	features: layering, exclusive-lock, deep-flatten
	op_features: 
	flags: 
	create_timestamp: Fri Mar 10 08:06:24 2023
	access_timestamp: Fri Mar 10 08:06:24 2023
	modify_timestamp: Fri Mar 10 08:21:29 2023
	mirroring state: enabled
	mirroring mode: snapshot
	mirroring global id: 3d66e921-1c82-46cb-a9e9-164cbdac5fba
	mirroring primary: true

after resync from secondary
[ceph: root@ceph-rbd2-sangadi-udsc8d-node1-installer /]# rbd info rep_pool_DqufdZqgib/rep_image_WEbRXwjgSo
rbd image 'rep_image_WEbRXwjgSo':
	size 1 GiB in 256 objects
	order 22 (4 MiB objects)
	snapshot_count: 1
	id: 3a197fb4a96c
	block_name_prefix: rbd_data.3a197fb4a96c
	format: 2
	features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, non-primary
	op_features: 
	flags: 
	create_timestamp: Fri Mar 10 08:19:25 2023
	access_timestamp: Fri Mar 10 08:19:25 2023
	modify_timestamp: Fri Mar 10 08:19:25 2023
	mirroring state: enabled
	mirroring mode: snapshot
	mirroring global id: 3d66e921-1c82-46cb-a9e9-164cbdac5fba
	mirroring primary: false
[ceph: root@ceph-rbd2-sangadi-udsc8d-node1-installer /]# rbd mirror image resync rep_pool_DqufdZqgib/rep_image_WEbRXwjgSo
Flagged image for resync from primary
[ceph: root@ceph-rbd2-sangadi-udsc8d-node1-installer /]# 
[ceph: root@ceph-rbd2-sangadi-udsc8d-node1-installer /]# rbd info rep_pool_DqufdZqgib/rep_image_WEbRXwjgSo
rbd image 'rep_image_WEbRXwjgSo':
	size 1 GiB in 256 objects
	order 22 (4 MiB objects)
	snapshot_count: 1
	id: 3a197d310212
	block_name_prefix: rbd_data.3a197d310212
	format: 2
	features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, non-primary
	op_features: 
	flags: object map invalid, fast diff invalid
	create_timestamp: Fri Mar 10 08:32:33 2023
	access_timestamp: Fri Mar 10 08:32:33 2023
	modify_timestamp: Fri Mar 10 08:32:33 2023
	mirroring state: enabled
	mirroring mode: snapshot
	mirroring global id: 3d66e921-1c82-46cb-a9e9-164cbdac5fba
	mirroring primary: false

Once `flags` says object map invalid, fast diff invalid then these image features should be removed from `features` section of image info. 

Version-Release number of selected component (if applicable):
ceph version 16.2.10-138.el8cp (a63ae467c8e1f7503ea3855893f1e5ca189a71b9) pacific (stable)

How reproducible:
Always

Steps to Reproduce:
1.Configure two way mirroring with snapshot based
2.disable some image feature from primary
3.after resync from secondary check features got disabled in features section as well other than flags section.

Because in journal based mirroring once these features removed from primary it also getting removed from secondary in the image info feature section.

Hence requesting to removed invalid features from features section along with flags retained.

Actual results:
Invalid image features kept as it is in `features` section of image info.

Expected results:
Remove invalid image feature from `features` section of image info.

Additional info: