Description of problem (please be detailed as possible and provide log snippests): Deploymnet Type: AZURE IPI FIPS ENCRYPTION 1AZ RHCOS 3M 3W ocs-storagecluster is in progressing state due to noobaa in configuring state due to tls: failed to verify certificate: x509: certificate signed by unknown authority Version of all relevant components (if applicable): ocs-registry:4.15.0-144 Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Yes Is there any workaround available to the best of your knowledge? No Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 1 Can this issue reproducible? 2/2 Can this issue reproduce from the UI? Not tried If this is a regression, please provide more details to justify this: Yes Steps to Reproduce: 1. install ODF using ocs-ci and check storagecluster status Actual results: storagecluster status: Status: Conditions: Last Heartbeat Time: 2024-02-19T09:33:36Z Last Transition Time: 2024-02-19T09:33:36Z Message: Version check successful Reason: VersionMatched Status: False Type: VersionMismatch Last Heartbeat Time: 2024-02-19T09:40:39Z Last Transition Time: 2024-02-19T09:38:15Z Message: Reconcile completed successfully Reason: ReconcileCompleted Status: True Type: ReconcileComplete Last Heartbeat Time: 2024-02-19T09:38:12Z Last Transition Time: 2024-02-19T09:38:12Z Message: Reconcile completed successfully Reason: ReconcileCompleted Status: True Type: Available Last Heartbeat Time: 2024-02-19T09:40:39Z Last Transition Time: 2024-02-19T09:38:13Z Message: Waiting on Nooba instance to finish initialization Reason: NoobaaInitializing Status: True Type: Progressing Last Heartbeat Time: 2024-02-19T09:38:12Z Last Transition Time: 2024-02-19T09:38:12Z Message: Reconcile completed successfully Reason: ReconcileCompleted Status: False Type: Degraded Last Heartbeat Time: 2024-02-19T09:38:40Z Last Transition Time: 2024-02-19T09:38:13Z Message: CephCluster is creating: Processing OSD 2 on PVC "ocs-deviceset-managed-csi-0-data-0gnxqg" Reason: ClusterStateCreating Status: False Type: Upgradeable Current Mon Count: 3 Failure Domain: rack Failure Domain Key: topology.rook.io/rack . . . Phase: Progressing Expected results: storagecluster should be in Ready state Additional info: $ oc get noobaa NAME S3-ENDPOINTS STS-ENDPOINTS IMAGE PHASE AGE noobaa ["https://10.0.128.5:30107"] ["https://10.0.128.5:30373"] registry.redhat.io/odf4/mcg-core-rhel9@sha256:bfd4cb6d5c7526fdc8f35d4fedb1ff9ecb39ac7564c8f05c2d5e12aea41048e9 Configuring 19h $ oc get noobaa noobaa -o yaml apiVersion: noobaa.io/v1alpha1 kind: NooBaa metadata: creationTimestamp: "2024-02-19T09:37:36Z" finalizers: - noobaa.io/graceful_finalizer generation: 1 labels: app: noobaa name: noobaa namespace: openshift-storage ownerReferences: - apiVersion: ocs.openshift.io/v1 blockOwnerDeletion: true controller: true kind: StorageCluster name: ocs-storagecluster uid: d4c313f2-e6e6-4efc-9bb1-6af0b720bd21 resourceVersion: "633764" uid: 388e1f36-bc7b-451e-b2fc-70b6e064b793 . . . status: accounts: admin: secretRef: name: noobaa-admin namespace: openshift-storage actualImage: registry.redhat.io/odf4/mcg-core-rhel9@sha256:bfd4cb6d5c7526fdc8f35d4fedb1ff9ecb39ac7564c8f05c2d5e12aea41048e9 conditions: - lastHeartbeatTime: "2024-02-20T04:33:08Z" lastTransitionTime: "2024-02-19T09:37:36Z" message: 'failed to start creating storage account: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/9bef6367-8ff5-4f08-84c9-3da195c53762/resourceGroups/j-156zife1c33-d-b9t2d-rg/providers/Microsoft.Storage/storageAccounts/noobaaaccountssm3a?api-version=2019-06-01: StatusCode=0 -- Original Error: adal: Failed to execute the refresh request. Error = ''Post "https://login.microsoftonline.com/9cf78105-e3e9-4321-b88d-b001b66c762b/oauth2/token?api-version=1.0": tls: failed to verify certificate: x509: certificate signed by unknown authority''' reason: TemporaryError status: "False" type: Available - lastHeartbeatTime: "2024-02-20T04:33:08Z" lastTransitionTime: "2024-02-19T09:37:36Z" message: 'failed to start creating storage account: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/9bef6367-8ff5-4f08-84c9-3da195c53762/resourceGroups/j-156zife1c33-d-b9t2d-rg/providers/Microsoft.Storage/storageAccounts/noobaaaccountssm3a?api-version=2019-06-01: StatusCode=0 -- Original Error: adal: Failed to execute the refresh request. Error = ''Post "https://login.microsoftonline.com/9cf78105-e3e9-4321-b88d-b001b66c762b/oauth2/token?api-version=1.0": tls: failed to verify certificate: x509: certificate signed by unknown authority''' reason: TemporaryError status: "True" type: Progressing - lastHeartbeatTime: "2024-02-20T04:33:08Z" lastTransitionTime: "2024-02-19T09:37:36Z" message: 'failed to start creating storage account: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/9bef6367-8ff5-4f08-84c9-3da195c53762/resourceGroups/j-156zife1c33-d-b9t2d-rg/providers/Microsoft.Storage/storageAccounts/noobaaaccountssm3a?api-version=2019-06-01: StatusCode=0 -- Original Error: adal: Failed to execute the refresh request. Error = ''Post "https://login.microsoftonline.com/9cf78105-e3e9-4321-b88d-b001b66c762b/oauth2/token?api-version=1.0": tls: failed to verify certificate: x509: certificate signed by unknown authority''' reason: TemporaryError status: "False" type: Degraded - lastHeartbeatTime: "2024-02-20T04:33:08Z" lastTransitionTime: "2024-02-19T09:37:36Z" message: 'failed to start creating storage account: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/9bef6367-8ff5-4f08-84c9-3da195c53762/resourceGroups/j-156zife1c33-d-b9t2d-rg/providers/Microsoft.Storage/storageAccounts/noobaaaccountssm3a?api-version=2019-06-01: StatusCode=0 -- Original Error: adal: Failed to execute the refresh request. Error = ''Post "https://login.microsoftonline.com/9cf78105-e3e9-4321-b88d-b001b66c762b/oauth2/token?api-version=1.0": tls: failed to verify certificate: x509: certificate signed by unknown authority''' reason: TemporaryError status: "False" type: Upgradeable - lastHeartbeatTime: "2024-02-20T04:25:37Z" lastTransitionTime: "2024-02-19T09:37:37Z" status: k8s type: KMS-Type - lastHeartbeatTime: "2024-02-20T04:25:37Z" lastTransitionTime: "2024-02-19T09:37:38Z" status: Sync type: KMS-Status observedGeneration: 1 phase: Configuring must gather logs: https://url.corp.redhat.com/45bc1ca job: https://url.corp.redhat.com/cc698ab
verified with build: ocs-registry:4.15.0-155 job: https://url.corp.redhat.com/b896aca logs: https://url.corp.redhat.com/745a5b4 2024-03-04 20:54:42 15:24:42 - MainThread - ocs_ci.ocs.resources.storage_cluster - INFO - Verifying status of storage cluster: ocs-storagecluster 2024-03-04 20:54:42 15:24:42 - MainThread - ocs_ci.ocs.resources.storage_cluster - INFO - Check if StorageCluster: ocs-storagecluster is in Succeeded phase 2024-03-04 20:54:42 15:24:42 - MainThread - ocs_ci.utility.utils - INFO - Executing command: oc --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig -n openshift-storage get StorageCluster ocs-storagecluster -n openshift-storage -o yaml 2024-03-04 20:54:42 15:24:42 - MainThread - ocs_ci.ocs.ocp - INFO - Resource ocs-storagecluster is in phase: Ready!
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.15.0 security, enhancement, & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:1383