Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2069480

Summary: Update Status Update to 4.8.33 in progress hung for 2 days
Product: OpenShift Container Platform Reporter: kevin <schunhua>
Component: Cluster Version OperatorAssignee: Over the Air Updates <aos-team-ota>
Status: CLOSED DUPLICATE QA Contact: liujia <jiajliu>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.8CC: aos-bugs, wking
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-30 16:59:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Update Status Update to 4.8.33 in progress hung for 2 days
none
can not select upgrade to 4.7 version
none
update history none

Description kevin 2022-03-29 02:45:53 UTC
Created attachment 1868878 [details]
Update Status Update to 4.8.33 in progress hung for 2 days

we are upgrading to the Openshift cluster to 4.8.33, but it in progress about 2 days, seems hung. 

Update Status Update to 4.8.33 in progress hung for 2 days

Comment 1 W. Trevor King 2022-03-29 02:51:20 UTC
Screenshot showing 0% ClusterOperator with happy MachineConfigPools isn't all that useful.  Can you point us at a must-gather or something with more detail?  Concerning detail from the screenshot is the last completed version being 4.6.48.  Jumping straight from 4.6 to 4.8 is not supported; you need to update one 4.y minor at a time (which is a requirement we pass through from upstream Kubernetes components).

Comment 2 kevin 2022-03-29 02:56:03 UTC
Created attachment 1868892 [details]
can not select upgrade to 4.7 version

can not select upgrade to 4.7 version, do you know the reason?

Comment 3 liujia 2022-03-29 07:52:52 UTC
> Created attachment 1868892 [details]
can not select upgrade to 4.7 version

If the cluster is on v4.6.48 correctly, the available channels should not be like this(only 4.8 and 4.9 channels). I suspect there maybe an ongoing upgrade to v4.7 while user changed channel to stable-4.8 at that time. Then due to https://bugzilla.redhat.com/show_bug.cgi?id=1802553, users could trigger a new upgrade to v4.8 from web-console while there was an upgrade ongoing(unexpected). And this is not allowed from cli.

Could you check is there a Partial upgrade to v4.7 from `Update History`?

Comment 4 kevin 2022-03-29 08:21:48 UTC
Created attachment 1868928 [details]
update history

update history

Comment 5 kevin 2022-03-29 08:22:30 UTC
yes, i checked the history, there is 4.7. but what I should do now, seems it is hung here about 3 days.

Comment 6 liujia 2022-03-29 08:50:29 UTC
Hmm, now only snapshots could not help identifying the state of the update. I mean we don't know if the update is ongoing or not. So we need must-gather logs or at least cvo log to help dev checking it further.

Comment 7 kevin 2022-03-29 09:04:56 UTC
is there way or command to stop or kill current install? 
is there command to check if it is hung? 
how to collect the logs?

Thank you very much!

Comment 8 W. Trevor King 2022-03-29 19:09:24 UTC
Docs for gathering data, including must-gathers under [1].

[1]: https://docs.openshift.com/container-platform/4.8/support/gathering-cluster-data.html

Comment 9 W. Trevor King 2022-03-29 19:15:08 UTC
But from the comment 4 screenshot, the partial 4.7.44 started and terminated at the same time (to the minute).  So I'm closing this as a dup of bug 1802553.  If your cluster is subscribed and you feel like you need help with recovery, probably open a support case [1], although they'll likely want a must-gather as well.

[1]: https://docs.openshift.com/container-platform/4.8/support/getting-support.html#support-submitting-a-case_getting-support

*** This bug has been marked as a duplicate of bug 1802553 ***

Comment 10 kevin 2022-03-30 01:18:21 UTC
thanks for the info:

is there way or command to stop or kill current install with force option? so that we can continue a new upgrade?


Thank you very much!

Comment 11 W. Trevor King 2022-03-30 16:59:18 UTC
Bug 1802553 is about putting some guard-rails in place to make it harder to get into this situation in the future.  If you want support assistance for recovering your particular cluster, you should open a support case, following the process linked from comment 9.

*** This bug has been marked as a duplicate of bug 1802553 ***