Bug 1958373 - OCS 4.8 deployment fails since 4.8.0-378.ci with storagecluster stuck in Progressing state
Summary: OCS 4.8 deployment fails since 4.8.0-378.ci with storagecluster stuck in Prog...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenShift Container Storage
Classification: Red Hat Storage
Component: Multi-Cloud Object Gateway
Version: 4.8
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: OCS 4.8.0
Assignee: Jacky Albo
QA Contact: Neha Berry
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-07 18:37 UTC by Neha Berry
Modified: 2021-08-03 18:16 UTC (History)
4 users (show)

Fixed In Version: v4.8.0-386.ci
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-08-03 18:16:11 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github noobaa noobaa-operator pull 630 0 None open Undoing unneeded change to container security context 2021-05-09 08:27:53 UTC
Red Hat Product Errata RHBA-2021:3003 0 None None None 2021-08-03 18:16:26 UTC

Description Neha Berry 2021-05-07 18:37:32 UTC
Description of problem (please be detailed as possible and provide log
snippests):
========================================================================
Note: We had a similar deployment failure in Bug 1952344 - OCS 4.8: v4.8.0-359 - storagecluster is in progressing state which was fixed with version 4.6.0-361.ci 

The last stable build of 4.8 where deployment succeeded is 4.8.0-374.ci. Since then, OCS deployment fails with CSV in Progressing state.

Creating a bug with details from the latest 4.8.0-382.ci build:
---------------------------------------------------------------------
2:50:07 - MainThread - ocs_ci.ocs.ocp - INFO - Resource ocs-operator.v4.8.0-382.ci is in phase: Succeeded!

13:36:38 - MainThread - ocs_ci.utility.utils - INFO - Executing command: oc -n openshift-storage get StorageCluster ocs-storagecluster -n openshift-storage -o yaml
13:36:39 - MainThread - ocs_ci.ocs.ocp - INFO - Resource ocs-storagecluster is in phase: Progressing!

# oc describe storagecluster
Last Heartbeat Time:   2021-05-06T17:38:27Z
    Last Transition Time:  2021-05-06T16:52:56Z
    Message:               Waiting on Nooba instance to finish initialization
    Reason:                NoobaaInitializing
    Status:                True
    Type:                  Progressing


Job links
---------------------

console logs =[1] = https://storage-jenkins-csb-ceph.cloud.paas.psi.redhat.com/job/ocs-
ci/381/console
[2] - https://storage-jenkins-csb-ceph.cloud.paas.psi.redhat.com/job/ocs-ci/378/console

Chat thread link - https://chat.google.com/room/AAAAREGEba8/yUKy4wDIDgY

Log location = https://storage-jenkins-csb-ceph.cloud.paas.psi.redhat.com/job/ocs-ci/381/artifact/logs/failed_testcase_ocs_logs_1620319669/test_deployment_ocs_logs/ocs_must_gather/




Version of all relevant components (if applicable):
=======================================================
openshift installer (4.8.0-0.nightly-2021-04-30-201824)
quay.io/rhceph-dev/ocs-registry:4.8.0-382.ci (4.8.0-378.ci)


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
=============================================================
Yes. Deployment fails

Is there any workaround available to the best of your knowledge?
====================================================================
Not sure

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
============================================================================
3

Can this issue reproducible?
==============================
Yes

Can this issue reproduce from the UI?
=========================================
Did not test

If this is a regression, please provide more details to justify this:
======================================================================
Yes

Steps to Reproduce:
=======================
1.Install OCS using ocs-ci
2.verify storage cluster is in ready state

Actual results
===================
storage cluster is in progressing state

Expected results:
====================
storage cluster should be in succeed phase

Additional info:
========================
oc describe storagecluster = https://storage-jenkins-csb-ceph.cloud.paas.psi.redhat.com/job/ocs-ci/381/artifact/logs/failed_testcase_ocs_logs_1620319669/test_deployment_ocs_logs/ocs_must_gather/quay-io-rhceph-dev-ocs-must-gather-sha256-1d34a430aee86cca33aa19a50d27bffa520a41d6490044563f601bb15d66e888/namespaces/openshift-storage/oc_output/storagecluster/*view*/



Initial analysis from the chat thread
==========================================


ocs-operator is waiting for NooBaa CR: Waiting on Nooba instance to finish initialization
NooBaa CR status: First endpoint is not ready yet
noobaa-endpoint Deployment is not able to create the noobaa-endpoint Pod:
'pods "noobaa-endpoint-65dc8bb596-" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, spec.containers[0].securityContext.capabilities.add: Invalid value: "SETGID": capability may not be added, spec.containers[0].securityContext.capabilities.add: Invalid value: "SETUID": capability may not be added, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "noobaa-endpoint": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, provider "rook-ceph": Forbidden: not usable by user or serviceaccount, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount, provider "rook-ceph-csi": Forbidden: not usable by user or serviceaccount]'

Comment 9 errata-xmlrpc 2021-08-03 18:16:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenShift Container Storage 4.8.0 container images bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3003


Note You need to log in before you can comment on or make changes to this bug.