Bug 2361474
| Summary: | Renamed mirror group on secondary cluster is not reflected on primary side and a subsequent planned failover results in up+error state | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Chaitanya <cdommeti> |
| Component: | RBD-Mirror | Assignee: | Vinay Bhaskar <vvarada> |
| Status: | CLOSED ERRATA | QA Contact: | Chaitanya <cdommeti> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 8.1 | CC: | ceph-eng-bugs, cephqe-warriors, idryomov, sangadi |
| Target Milestone: | --- | ||
| Target Release: | 9.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | ceph-20.1.0-16 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2026-01-29 06:55:26 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Red Hat Ceph Storage 9.0 Security and Enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2026:1536 |
Description of problem: Mirrored Group renaming is successful on secondary side. (Not sure if group renaming is supposed to be blocked on secondary side). The new name is reflected on secondary side but not on primary. A subsequent demote on primary and promote on secondary resulted the cluster in up+error state (description: bootstrap failed) Version-Release number of selected component (if applicable): 19.2.1-138.el9cp How reproducible: Always Steps to Reproduce: - Initial state of the cluster on secondary side: [root@ceph-rbd2-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1 g1: global_id: a8a05ecc-38d3-4d3d-95d5-439cbe9c827d state: up+replaying description: replaying, {"last_snapshot_bytes":0,"last_snapshot_complete_seconds":2,"local_snapshot_timestamp":1744984629,"remote_snapshot_timestamp":1744984629} service: ceph-rbd2-cd-cg126-dz8z92-node5.wnxcfv on ceph-rbd2-cd-cg126-dz8z92-node5 last_update: 2025-04-18 13:57:30 images: image: 57/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+replaying description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1744984629,"remote_snapshot_timestamp":1744984629,"replay_state":"idle"} peer_sites: name: ceph-rbd1 state: up+stopped description: last_update: 2025-04-18 13:57:15 images: image: 57/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+stopped description: local image is primary - Rename the group on secondary cluster and check the status on seconday as well as primary cluster. On secondary cluster the new name is reflected. However, on primary the group still has old name. root@ceph-rbd2-cd-cg126-dz8z92-node2 ~]# rbd group rename --pool p1 --group g1 --dest-pool p1 --dest-group g1_new [root@ceph-rbd2-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1 rbd: failed to get mirror info for group: (2) No such file or directory [root@ceph-rbd2-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1_new g1_new: global_id: a8a05ecc-38d3-4d3d-95d5-439cbe9c827d state: up+replaying description: replaying, {"last_snapshot_bytes":0,"last_snapshot_complete_seconds":2,"local_snapshot_timestamp":1744984629,"remote_snapshot_timestamp":1744984629} service: ceph-rbd2-cd-cg126-dz8z92-node5.wnxcfv on ceph-rbd2-cd-cg126-dz8z92-node5 last_update: 2025-04-18 14:01:00 images: image: 57/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+replaying description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1744984629,"remote_snapshot_timestamp":1744984629,"replay_state":"idle"} peer_sites: name: ceph-rbd1 state: up+stopped description: last_update: 2025-04-18 14:00:43 images: image: 57/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+stopped description: local image is primary [root@ceph-rbd1-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1_new rbd: failed to get mirror info for group: (2) No such file or directory root@ceph-rbd1-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1 g1: global_id: a8a05ecc-38d3-4d3d-95d5-439cbe9c827d state: up+stopped description: service: ceph-rbd1-cd-cg126-dz8z92-node5.ekkvnc on ceph-rbd1-cd-cg126-dz8z92-node5 last_update: 2025-04-18 14:02:13 images: image: 61/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+stopped description: local image is primary peer_sites: name: ceph-rbd2 state: up+replaying description: replaying, {"last_snapshot_bytes":0,"last_snapshot_complete_seconds":2,"local_snapshot_timestamp":1744984629,"remote_snapshot_timestamp":1744984629} last_update: 2025-04-18 14:02:30 images: image: 61/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+replaying description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1744984629,"remote_snapshot_timestamp":1744984629,"replay_state":"idle"} snapshots: .mirror.primary.a8a05ecc-38d3-4d3d-95d5-439cbe9c827d.1e6d23f9d8c48 - Now, the group(g1) is demoted on primary and group(g1_new) is promoted on secondary. This results in up_error state. [root@ceph-rbd1-cd-cg126-dz8z92-node2 ~]# rbd mirror group demote --pool p1 --group g1 Group demoted to non-primary [root@ceph-rbd1-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1 g1: global_id: a8a05ecc-38d3-4d3d-95d5-439cbe9c827d state: up+stopped description: service: ceph-rbd1-cd-cg126-dz8z92-node5.ekkvnc on ceph-rbd1-cd-cg126-dz8z92-node5 last_update: 2025-04-18 14:07:13 images: image: 61/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+stopped description: local image is primary peer_sites: name: ceph-rbd2 state: up+unknown description: remote group is non-primary last_update: 2025-04-18 14:07:30 images: [root@ceph-rbd2-cd-cg126-dz8z92-node2 ~]# rbd mirror group promote --pool p1 --group g1_new Group promoted to primary [root@ceph-rbd2-cd-cg126-dz8z92-node2 ~]# rbd mirror group demote --pool p1 --group g1_new Group demoted to non-primary [root@ceph-rbd2-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1_new g1_new: global_id: a8a05ecc-38d3-4d3d-95d5-439cbe9c827d state: up+error description: bootstrap failed service: ceph-rbd2-cd-cg126-dz8z92-node5.wnxcfv on ceph-rbd2-cd-cg126-dz8z92-node5 last_update: 2025-04-18 14:12:00 images: peer_sites: name: ceph-rbd1 state: up+unknown description: remote group is non-primary last_update: 2025-04-18 14:11:43 images: [root@ceph-rbd1-cd-cg126-dz8z92-node2 ~]# rbd mirror group status p1/g1 g1: global_id: a8a05ecc-38d3-4d3d-95d5-439cbe9c827d state: up+error description: bootstrap failed service: ceph-rbd1-cd-cg126-dz8z92-node5.ekkvnc on ceph-rbd1-cd-cg126-dz8z92-node5 last_update: 2025-04-18 14:11:13 images: peer_sites: name: ceph-rbd2 state: up+stopped description: last_update: 2025-04-18 14:11:00 images: image: 61/a1660cdd-f1ef-4c02-87c2-9de0148d8074 state: up+stopped description: local image is primary Actual results: New group name is not reflected on primary side and the status of group after failover operations is not proper. Expected results: New group name should be reflected on primary side if group renaming is allowed on secondary cluster and the subsequent failover operations should show proper status Additional info: