Bug 2338125 - Ceph upgrade from 7.1z3 to 8.0z2 fails with "Error: UPGRADE_REDEPLOY_DAEMON"
Summary: Ceph upgrade from 7.1z3 to 8.0z2 fails with "Error: UPGRADE_REDEPLOY_DAEMON"
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Cephadm
Version: 8.0
Hardware: Unspecified
OS: Linux
unspecified
urgent
Target Milestone: ---
: 8.0z2
Assignee: Adam King
QA Contact: Chaitanya
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2025-01-15 06:49 UTC by Chaitanya
Modified: 2025-03-06 14:21 UTC (History)
8 users (show)

Fixed In Version: ceph-19.2.0-82.el9cp
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2025-03-06 14:21:15 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-10457 0 None None None 2025-01-15 06:52:21 UTC
Red Hat Product Errata RHBA-2025:2457 0 None None None 2025-03-06 14:21:18 UTC

Description Chaitanya 2025-01-15 06:49:57 UTC
Description of problem:
Did fresh install of 7.1z3. Configured iSCSI gateway, target and intiator. Able to discover the target from the client. Enable iSCSI for dashboard. After verifying iSCSI configuration is fine on 7.1z3, tried to upgrade to 8.0z3 following the below steps:

 - ceph osd set noout 
 - ceph osd set noscrub
 - ceph osd set nodeep-scrub
 - ceph cephadm registry-login cp.stg.icr.io **
 - ceph config set mgr mgr/cephadm/container_image_grafana cp.stg.icr.io/cp/ibm- 
   ceph/grafana-rhel9:10.4.8-6
 - ceph config set mgr mgr/cephadm/container_image_keepalived cp.stg.icr.io/cp/ibm- 
   ceph/keepalived-rhel9:2.2.8-36
 - ceph config set mgr mgr/cephadm/container_image_haproxy cp.stg.icr.io/cp/ibm-ceph/haproxy-rhel9:2.4.22-38
 - ceph config set mgr mgr/cephadm/container_image_prometheus cp.stg.icr.io/cp/ibm-ceph/prometheus:v4.15.0-202412181437.p0.g1b43998.assembly.stream.el8
 - ceph config set mgr mgr/cephadm/container_image_node_exporter cp.stg.icr.io/cp/ibm-ceph/prometheus-node-exporter:v4.15.0- 
   202412170135.p0.gaed837c.assembly.stream.el8
 - ceph config set mgr mgr/cephadm/container_image_alertmanager cp.stg.icr.io/cp/ibm-ceph/prometheus-alertmanager:v4.15.0
   -202412170135.p0.g870ade5.assembly.stream.el8
 - ceph config set mgr mgr/cephadm/container_image_promtail cp.stg.icr.io/cp/ibm-ceph/promtail-rhel9:v3.0.0-9
 - ceph config set mgr mgr/cephadm/container_image_nvmeof cp.stg.icr.io/cp/ibm-ceph/nvmeof-rhel9:1.3.3-17
 - ceph config set mgr mgr/cephadm/container_image_loki  cp.stg.icr.io/cp/ibm-ceph/logging-loki-rhel8:v3.1.0-15
 - ceph orch upgrade check cp.stg.icr.io/cp/ibm-ceph/ceph-8-rhel9:8-64
 - ceph orch upgrade start cp.stg.icr.io/cp/ibm-ceph/ceph-8-rhel9:8-64

 Upgrade failed at the step of upgrading iSCSI daemon with message "Error: UPGRADE_REDEPLOY_DAEMON: as seen from "ceph orch upgrade status"


How reproducible:
 Always

Steps to Reproduce:
1. Fresh install 7.1z3
2. Configure iSCSI
3. Upgrade to 8.0z2

Actual results:
 Upgrade failed with error : "Error: UPGRADE_REDEPLOY_DAEMON: as seein from "ceph orch upgrade status"

[ceph: root@ceph-cdiscsi-k5nigl-node1-installer /]# ceph orch upgrade status
{
    "target_image": "cp.stg.icr.io/cp/ibm-ceph/ceph-8-rhel9@sha256:42aeb195cd3c284e1f350922611a68a6f61c394f5b6b4ec9172165fb59b18153",
    "in_progress": true,
    "which": "Upgrading all daemon types on all hosts",
    "services_complete": [
        "mon",
        "osd",
        "mgr"
    ],
    "progress": "14/16 daemons upgraded",
    "message": "Error: UPGRADE_REDEPLOY_DAEMON: Upgrading daemon iscsi.iscsi.ceph-cdiscsi-k5nigl-node1-installer.wradkz on host ceph-cdiscsi-k5nigl-node1-installer failed.",
    "is_paused": true
}


Expected results:
 
Upgrade should have been sucessful

Additional info:
Also in the output of "ceph orch upgrade status", 'in_progress' is 'true' and 'is_paused' is also 'true', which is confusing. 
If the upgrade failed, should the 'in_progress' field be set to 'false' ?

Comment 28 errata-xmlrpc 2025-03-06 14:21:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 8.0 security, bug fixes, and enhancement updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2025:2457


Note You need to log in before you can comment on or make changes to this bug.