Bug 2338125
| Summary: | Ceph upgrade from 7.1z3 to 8.0z2 fails with "Error: UPGRADE_REDEPLOY_DAEMON" | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Chaitanya <cdommeti> |
| Component: | Cephadm | Assignee: | Adam King <adking> |
| Status: | CLOSED ERRATA | QA Contact: | Chaitanya <cdommeti> |
| Severity: | urgent | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 8.0 | CC: | acaro, adking, ceph-eng-bugs, cephqe-warriors, idryomov, mmurthy, sunnagar, tserlin |
| Target Milestone: | --- | ||
| Target Release: | 8.0z2 | ||
| Hardware: | Unspecified | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | ceph-19.2.0-82.el9cp | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2025-03-06 14:21:15 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat Ceph Storage 8.0 security, bug fixes, and enhancement updates), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2025:2457 |
Description of problem: Did fresh install of 7.1z3. Configured iSCSI gateway, target and intiator. Able to discover the target from the client. Enable iSCSI for dashboard. After verifying iSCSI configuration is fine on 7.1z3, tried to upgrade to 8.0z3 following the below steps: - ceph osd set noout - ceph osd set noscrub - ceph osd set nodeep-scrub - ceph cephadm registry-login cp.stg.icr.io ** - ceph config set mgr mgr/cephadm/container_image_grafana cp.stg.icr.io/cp/ibm- ceph/grafana-rhel9:10.4.8-6 - ceph config set mgr mgr/cephadm/container_image_keepalived cp.stg.icr.io/cp/ibm- ceph/keepalived-rhel9:2.2.8-36 - ceph config set mgr mgr/cephadm/container_image_haproxy cp.stg.icr.io/cp/ibm-ceph/haproxy-rhel9:2.4.22-38 - ceph config set mgr mgr/cephadm/container_image_prometheus cp.stg.icr.io/cp/ibm-ceph/prometheus:v4.15.0-202412181437.p0.g1b43998.assembly.stream.el8 - ceph config set mgr mgr/cephadm/container_image_node_exporter cp.stg.icr.io/cp/ibm-ceph/prometheus-node-exporter:v4.15.0- 202412170135.p0.gaed837c.assembly.stream.el8 - ceph config set mgr mgr/cephadm/container_image_alertmanager cp.stg.icr.io/cp/ibm-ceph/prometheus-alertmanager:v4.15.0 -202412170135.p0.g870ade5.assembly.stream.el8 - ceph config set mgr mgr/cephadm/container_image_promtail cp.stg.icr.io/cp/ibm-ceph/promtail-rhel9:v3.0.0-9 - ceph config set mgr mgr/cephadm/container_image_nvmeof cp.stg.icr.io/cp/ibm-ceph/nvmeof-rhel9:1.3.3-17 - ceph config set mgr mgr/cephadm/container_image_loki cp.stg.icr.io/cp/ibm-ceph/logging-loki-rhel8:v3.1.0-15 - ceph orch upgrade check cp.stg.icr.io/cp/ibm-ceph/ceph-8-rhel9:8-64 - ceph orch upgrade start cp.stg.icr.io/cp/ibm-ceph/ceph-8-rhel9:8-64 Upgrade failed at the step of upgrading iSCSI daemon with message "Error: UPGRADE_REDEPLOY_DAEMON: as seen from "ceph orch upgrade status" How reproducible: Always Steps to Reproduce: 1. Fresh install 7.1z3 2. Configure iSCSI 3. Upgrade to 8.0z2 Actual results: Upgrade failed with error : "Error: UPGRADE_REDEPLOY_DAEMON: as seein from "ceph orch upgrade status" [ceph: root@ceph-cdiscsi-k5nigl-node1-installer /]# ceph orch upgrade status { "target_image": "cp.stg.icr.io/cp/ibm-ceph/ceph-8-rhel9@sha256:42aeb195cd3c284e1f350922611a68a6f61c394f5b6b4ec9172165fb59b18153", "in_progress": true, "which": "Upgrading all daemon types on all hosts", "services_complete": [ "mon", "osd", "mgr" ], "progress": "14/16 daemons upgraded", "message": "Error: UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.iscsi.ceph-cdiscsi-k5nigl-node1-installer.wradkz on host ceph-cdiscsi-k5nigl-node1-installer failed.", "is_paused": true } Expected results: Upgrade should have been sucessful Additional info: Also in the output of "ceph orch upgrade status", 'in_progress' is 'true' and 'is_paused' is also 'true', which is confusing. If the upgrade failed, should the 'in_progress' field be set to 'false' ?