Description of the problem: Attempting to install RHACM 2.5.0-DOWNSTREAM-2022-03-01-18-33-27 mutliclusterhub is stuck in installing. looking at the components see: oc get mch multiclusterhub -n rhacm -o json | jq '.status.components | map(select(.status == "Unknown"))' [ { "lastTransitionTime": "2022-03-02T11:09:52Z", "message": "Registration agent stopped updating its lease.", "reason": "ManagedClusterLeaseUpdateStopped", "status": "Unknown", "type": "ManagedClusterConditionAvailable" } ] Further klusterlet CRD is not created: oc get klusterlets klusterlet -o yaml error: the server doesn't have a resource type "klusterlets" Operator snapshot version: 2.5.0-DOWNSTREAM-2022-03-01-18-33-27 OCP version: 4.10.0-0.nightly-2022-02-26-230022 Steps to reproduce: 1. Attempt to install RHACM 2.5.0-DOWNSTREAM-2022-03-01-18-33-27 Actual results: mutliclusterhub is stuck in installing Expected results: mutliclusterhub to complete installation Additional info: deleting local-cluster-import secret to trigger reconcile allows mutliclusterhub to complete installation
reproducing with: OCP: 4.10.0-0.nightly-2022-03-16-041611 2.5.0-DOWNSTREAM-2022-03-14-18-18-07 oc get mch NAME STATUS AGE multiclusterhub Installing 3m45s oc get mch NAME STATUS AGE multiclusterhub Installing 3m45s [root@sealusa34 ~]# oc describe mch multiclusterhub Name: multiclusterhub Namespace: rhacm Labels: <none> Annotations: <none> API Version: operator.open-cluster-management.io/v1 Kind: MultiClusterHub Metadata: Creation Timestamp: 2022-03-17T02:04:23Z Finalizers: finalizer.operator.open-cluster-management.io Generation: 3 Managed Fields: API Version: operator.open-cluster-management.io/v1 Fields Type: FieldsV1 fieldsV1: f:spec: .: f:imagePullSecret: Manager: kubectl-create Operation: Update Time: 2022-03-17T02:04:23Z API Version: operator.open-cluster-management.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"finalizer.operator.open-cluster-management.io": f:spec: f:availabilityConfig: f:components: f:enableClusterBackup: f:ingress: .: f:sslCiphers: f:separateCertificateManagement: Manager: multiclusterhub-operator Operation: Update Time: 2022-03-17T02:04:32Z API Version: operator.open-cluster-management.io/v1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:components: .: f:application-chart-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:cluster-lifecycle-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:console-chart-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:grc-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:local-cluster: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:management-ingress-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:multicluster-engine: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:multicluster-engine-csv: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:multicluster-engine-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:multiclusterhub-repo: .: f:lastTransitionTime: f:reason: f:status: f:type: f:policyreport-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:search-prod-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:volsync-addon-controller-sub: .: f:lastTransitionTime: f:message: f:reason: f:status: f:type: f:conditions: f:desiredVersion: f:phase: Manager: multiclusterhub-operator Operation: Update Subresource: status Time: 2022-03-17T02:06:15Z Resource Version: 302497 UID: 87ac28ab-d5bb-4cd8-acff-1258c8d7334f Spec: Availability Config: High Components: Enabled: true Name: multiclusterhub-repo Enabled: true Name: search Enabled: true Name: management-ingress Enabled: true Name: application-ui Enabled: true Name: console Enabled: true Name: insights Enabled: true Name: grc Enabled: true Name: cluster-lifecycle Enabled: true Name: volsync Enabled: true Name: multicluster-engine Enabled: false Name: cluster-backup Enabled: false Name: cluster-proxy-addon Enable Cluster Backup: false Image Pull Secret: pull-secret Ingress: Ssl Ciphers: ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-GCM-SHA256 Separate Certificate Management: false Status: Components: Application - Chart - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Cluster - Lifecycle - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Console - Chart - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Grc - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Local - Cluster: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Management - Ingress - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Multicluster - Engine: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Multicluster - Engine - Csv: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Multicluster - Engine - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Multiclusterhub - Repo: Last Transition Time: 2022-03-17T02:05:00Z Reason: MinimumReplicasAvailable Status: True Type: Available Policyreport - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Search - Prod - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Volsync - Addon - Controller - Sub: Last Transition Time: 2022-03-17T02:02:51Z Message: No conditions available Reason: No conditions available Status: Unknown Type: Unknown Conditions: Last Transition Time: 2022-03-17T02:04:43Z Last Update Time: 2022-03-17T02:04:54Z Message: Created new resource Reason: NewResourceCreated Status: True Type: Progressing Desired Version: 2.5.0 Phase: Installing Events: <none>
multiclusterhub-operator log shows: 2022-03-17T02:28:55.522Z ERROR controller.multiclusterhub Reconciler error {"reconciler group": "operator.open-cluster-management.io", "reconciler kind": "MultiClusterHub", "name": "multiclusterhub", "namespace": "rhacm", "error": "CSV not located for subscription: multicluster-engine/multicluster-engine"}
the second issue is different from the original one. all components are Available in mch status in the original issue. we just finished dev sprints today. I think the codes changes are frequent in the past week. could you try to install the latest snapshot to verify.
reproduced with 2.5.0-DOWNSTREAM-2022-03-17-03-36-41
Hi @jqiu , can we get an outlook for resolution for this bug. Thx.
G2Bsync 1085295727 comment zhiweiyin318 Fri, 01 Apr 2022 01:21:54 UTC G2Bsync did we meet the issue in the latest snapshot? I think the issue is because the code changes frequently in the dev sprint as we discussed in slack.
Did not reproduce this with multiple redeploys of RHACM 2.5.0-DOWNSTREAM-2022-04-11-09-21-38
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat Advanced Cluster Management 2.5 security updates, images, and bug fixes), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:4956
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days