Bug 2310839 - [8.0 staggered Upgrade] : Upgrade of MGR in staggered approach also started upgrading NVMeoF service
Summary: [8.0 staggered Upgrade] : Upgrade of MGR in staggered approach also started u...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Cephadm
Version: 8.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 8.0
Assignee: Adam King
QA Contact: Mohit Bisht
URL:
Whiteboard:
Depends On: 2278778
Blocks: 2267614 2298578 2298579 2317218
TreeView+ depends on / blocked
 
Reported: 2024-09-09 12:46 UTC by Sunil Kumar Nagaraju
Modified: 2024-11-25 09:09 UTC (History)
7 users (show)

Fixed In Version: ceph-19.1.1-8.el9cp
Doc Type: Bug Fix
Doc Text:
.NVMe-oF daemons now upgrade after the MON daemons in staggered upgrade scenarios Previously, NVMe-oF depended on the version of the monitor daemon. As a result, in the previous staggered upgrade scenario, the NVMe-oF daemon would be upgraded as soon as the MGR daemons were upgraded. This placed the NVME-oF daemon before the MON daemons and caused the MON daemons to fail to deploy. With this fix, the upgrade of the NVMe-oF daemon is moved to the end of the upgrade order. NVMe-oF daemons now upgrade later in staggered upgrade scenarios, upgrading after the MON daemons and start up as expected.
Clone Of: 2278778
Environment:
Last Closed: 2024-11-25 09:09:18 UTC
Embargoed:
rlepaksh: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-9736 0 None None None 2024-09-09 12:50:36 UTC
Red Hat Product Errata RHBA-2024:10216 0 None None None 2024-11-25 09:09:30 UTC

Description Sunil Kumar Nagaraju 2024-09-09 12:46:14 UTC
+++ This bug was initially created as a clone of Bug #2278778 +++


Currently in 8.0 dev build, with MGR upgrade the NVMeoF GW daemons also started getting upgraded to new version i.e 1.3.1-1 and all are working.

But the issue is nvmeof gw upgrade are still not supported in staggered upgrade workflow, which was partially fixed in cloned BZ 2278778, where nvmeof upgrade happned post ceph-exporter.

Whats needed is have this upgrade workflow supported as we get from error comment,

mgr -> mon -> crash -> osd -> mds -> rgw -> rbd-mirror -> cephfs-mirror -> ceph-exporter -> iscsi -> nfs -> nvmeof


So upgrade should support upgrading of NVMeoF gateways independently. 



[ceph: root@ceph-sunilkumar-02-meya5y-node1-installer /]# ceph orch upgrade start --image cp.stg.icr.io/cp/ibm-ceph/ceph-8-rhel9:8-27  --daemon-types nvmeof
Error EINVAL: Cannot start upgrade. Daemons with types earlier in upgrade order than given types need upgrading.
Please first upgrade mon.ceph-sunilkumar-02-meya5y-node3, osd.1, osd.9, osd.8, osd.5, mon.ceph-sunilkumar-02-meya5y-node1-installer, mgr.ceph-sunilkumar-02-meya5y-node2.ciqsnw, mon.ceph-sunilkumar-02-meya5y-node2, osd.4, osd.7, mgr.ceph-sunilkumar-02-meya5y-node1-installer.uhyezu, osd.2, osd.3, osd.0, osd.11, osd.6, osd.10
NOTE: Enforced upgrade order is: mgr -> mon -> crash -> osd -> mds -> rgw -> rbd-mirror -> cephfs-mirror -> ceph-exporter -> iscsi -> nfs -> nvmeof

Comment 10 errata-xmlrpc 2024-11-25 09:09:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 8.0 security, bug fix, and enhancement updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:10216


Note You need to log in before you can comment on or make changes to this bug.