Bug 2123697

Summary: report-status-to-provider pod in CreateContainerError state on consumer
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Jilju Joy <jijoy>
Component: buildAssignee: Tamil <tmuthami>
Status: CLOSED ERRATA QA Contact: Jilju Joy <jijoy>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.11CC: aeyal, dbindra, dkhandel, kramdoss, madam, muagarwa, ocs-bugs, odf-bz-bot, sheggodu
Target Milestone: ---   
Target Release: ODF 4.11.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 2123724 (view as bug list) Environment:
Last Closed: 2022-09-14 15:15:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2120314, 2123724    

Description Jilju Joy 2022-09-02 10:56:49 UTC
Description of problem:
Many instances of the pod "report-status-to-provider" are remaining in the state "CreateContainerError" in the consumer cluster.

$ oc get pods
NAME                                                              READY   STATUS                 RESTARTS      AGE
4747b56e58799d351945d54657efb0b83a8ca1d00dd7e5380d06d3fa794tw4q   0/1     Completed              0             4h16m
6950ec9eb29172a76d40e4b2ff5e2376b711b37788e0686023b978d02f7xd9h   0/1     Completed              0             4h16m
a4d77567539501a711bc141b9ebab2e741b78999a50cb7cc1b157c395czh584   0/1     Completed              0             4h16m
addon-ocs-consumer-qe-catalog-7z42p                               1/1     Running                0             4h13m
alertmanager-managed-ocs-alertmanager-0                           2/2     Running                0             4h10m
alertmanager-managed-ocs-alertmanager-1                           2/2     Running                0             4h10m
alertmanager-managed-ocs-alertmanager-2                           2/2     Running                0             4h10m
b82aad7f30c569c3b7e910a94ee8198c9b387508021164ca4ac419b54f226pq   0/1     Completed              0             4h16m
b8bc02ee61adb95617b7822454117ffec980e791abef5220e3c66ee11bv86ml   0/1     Completed              0             4h16m
csi-addons-controller-manager-8455c9946d-8fq69                    2/2     Running                0             56m
csi-cephfsplugin-64xhk                                            3/3     Running                0             56m
csi-cephfsplugin-dxws8                                            3/3     Running                0             56m
csi-cephfsplugin-provisioner-5bb48774f9-2l9pb                     6/6     Running                0             56m
csi-cephfsplugin-provisioner-5bb48774f9-gwln2                     6/6     Running                0             56m
csi-cephfsplugin-rdzjh                                            3/3     Running                0             56m
csi-rbdplugin-2d4mc                                               4/4     Running                0             56m
csi-rbdplugin-688nf                                               4/4     Running                0             56m
csi-rbdplugin-7ms8d                                               4/4     Running                0             56m
csi-rbdplugin-provisioner-6987d55d87-57tfd                        7/7     Running                0             56m
csi-rbdplugin-provisioner-6987d55d87-wqcqn                        7/7     Running                0             56m
df4206ef9f6068058722977a3e2821d93212a70782e6a82a8a9e796903wdgzw   0/1     Completed              0             4h16m
ocs-metrics-exporter-76bd86987c-jtpns                             1/1     Running                0             56m
ocs-operator-684f88fcc7-79sqx                                     1/1     Running                0             57m
ocs-osd-controller-manager-545b5759db-glpk2                       3/3     Running                0             78m
odf-console-6c77d5df7f-zssq7                                      1/1     Running                0             57m
odf-operator-controller-manager-7c8fb545df-4p775                  2/2     Running                0             57m
prometheus-managed-ocs-prometheus-0                               3/3     Running                0             4h10m
prometheus-operator-8547cc9f89-2v4l2                              1/1     Running                0             4h15m
redhat-operators-s6x69                                            1/1     Running                0             4h13m
report-status-to-provider-27701857-fzvsl                          0/1     CreateContainerError   0             56m
report-status-to-provider-27701858-vgcq7                          0/1     CreateContainerError   0             55m
report-status-to-provider-27701859-xnwc9                          0/1     CreateContainerError   0             54m
report-status-to-provider-27701860-4m6nh                          0/1     CreateContainerError   0             53m
report-status-to-provider-27701861-srmjj                          0/1     CreateContainerError   0             52m
report-status-to-provider-27701862-8n7ds                          0/1     CreateContainerError   0             51m
report-status-to-provider-27701863-xmbvn                          0/1     CreateContainerError   0             50m
report-status-to-provider-27701864-sfx7c                          0/1     CreateContainerError   0             49m
report-status-to-provider-27701865-fccd5                          0/1     CreateContainerError   0             48m
report-status-to-provider-27701866-spn5h                          0/1     CreateContainerError   0             47m
report-status-to-provider-27701867-4rrcz                          0/1     CreateContainerError   0             46m
report-status-to-provider-27701868-5h4z8                          0/1     CreateContainerError   0             45m
report-status-to-provider-27701869-5shhf                          0/1     CreateContainerError   0             44m
report-status-to-provider-27701870-8sxr8                          0/1     CreateContainerError   0             43m
report-status-to-provider-27701871-22rjf                          0/1     CreateContainerError   0             42m
report-status-to-provider-27701872-fgx6n                          0/1     CreateContainerError   0             41m
report-status-to-provider-27701873-vgf7k                          0/1     CreateContainerError   0             40m
report-status-to-provider-27701874-pdllq                          0/1     CreateContainerError   0             39m
report-status-to-provider-27701875-ddjwh                          0/1     CreateContainerError   0             38m
report-status-to-provider-27701876-mk5wh                          0/1     CreateContainerError   0             37m
report-status-to-provider-27701877-5twxv                          0/1     CreateContainerError   0             36m
report-status-to-provider-27701878-x7ggm                          0/1     CreateContainerError   0             35m
report-status-to-provider-27701879-7s7zf                          0/1     CreateContainerError   0             34m
report-status-to-provider-27701880-6gv96                          0/1     CreateContainerError   0             33m
report-status-to-provider-27701881-hrszq                          0/1     CreateContainerError   0             32m
report-status-to-provider-27701882-9bpgp                          0/1     CreateContainerError   0             31m
report-status-to-provider-27701883-dwmv4                          0/1     CreateContainerError   0             30m
report-status-to-provider-27701884-gv7ps                          0/1     CreateContainerError   0             29m
report-status-to-provider-27701885-j4p7p                          0/1     CreateContainerError   0             28m
report-status-to-provider-27701886-7dmv8                          0/1     CreateContainerError   0             27m
report-status-to-provider-27701887-dmcjb                          0/1     CreateContainerError   0             26m
report-status-to-provider-27701888-pxm42                          0/1     CreateContainerError   0             25m
report-status-to-provider-27701889-g4fc6                          0/1     CreateContainerError   0             24m
report-status-to-provider-27701890-t9hql                          0/1     CreateContainerError   0             23m
report-status-to-provider-27701891-lzcts                          0/1     CreateContainerError   0             22m
report-status-to-provider-27701892-x8qmc                          0/1     CreateContainerError   0             21m
report-status-to-provider-27701893-bjwwn                          0/1     CreateContainerError   0             20m
report-status-to-provider-27701894-k4vsh                          0/1     CreateContainerError   0             19m
report-status-to-provider-27701895-gvmq6                          0/1     CreateContainerError   0             18m
report-status-to-provider-27701896-qf7nh                          0/1     CreateContainerError   0             17m
report-status-to-provider-27701897-mwlhz                          0/1     CreateContainerError   0             16m
report-status-to-provider-27701898-v7qbn                          0/1     CreateContainerError   0             15m
report-status-to-provider-27701899-vcbfs                          0/1     CreateContainerError   0             14m
report-status-to-provider-27701900-c55dr                          0/1     CreateContainerError   0             13m
report-status-to-provider-27701901-4snlz                          0/1     CreateContainerError   0             12m
report-status-to-provider-27701902-t5n7l                          0/1     CreateContainerError   0             11m
report-status-to-provider-27701903-znhrd                          0/1     CreateContainerError   0             10m
report-status-to-provider-27701904-99djk                          0/1     CreateContainerError   0             9m32s
report-status-to-provider-27701905-97bw7                          0/1     CreateContainerError   0             8m32s
report-status-to-provider-27701906-6bcg4                          0/1     CreateContainerError   0             7m32s
report-status-to-provider-27701907-hgbxl                          0/1     CreateContainerError   0             6m32s
report-status-to-provider-27701908-spwzp                          0/1     CreateContainerError   0             5m32s
report-status-to-provider-27701909-dfd8x                          0/1     CreateContainerError   0             4m32s
report-status-to-provider-27701910-rrfmx                          0/1     CreateContainerError   0             3m32s
report-status-to-provider-27701911-mv5pn                          0/1     CreateContainerError   0             2m32s
report-status-to-provider-27701912-wskz6                          0/1     CreateContainerError   0             92s
report-status-to-provider-27701913-lcpfm                          0/1     CreateContainerError   0             32s
rook-ceph-operator-85dc6665f-jrxpg                                1/1     Running                2 (56m ago)   57m
rook-ceph-tools-74cff48f58-b8k2n                                  1/1     Running                0             56m



ODF 4.11.1 was installed after uninstalling ODF 4.10.5 in both consumer and provider cluster. This was done because the upgrade to ODF 4.11.1 was not starting automatically(followed the same steps we were using to upgrade to ODF 4.11.0 from 4.10).


$ oc get managedocs -o yaml
apiVersion: v1
items:
- apiVersion: ocs.openshift.io/v1alpha1
  kind: ManagedOCS
  metadata:
    creationTimestamp: "2022-09-02T06:18:53Z"
    finalizers:
    - managedocs.ocs.openshift.io
    generation: 1
    name: managedocs
    namespace: openshift-storage
    resourceVersion: "452444"
    uid: 87bd030a-d780-4c60-adae-f004cc9ea421
  spec: {}
  status:
    components:
      alertmanager:
        state: Ready
      prometheus:
        state: Ready
      storageCluster:
        state: Ready
    reconcileStrategy: strict
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""


consumer must-gather - http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/jijoy-sep2-c1/jijoy-sep2-c1_20220902T054021/logs/testcases_1662114794/

provider must-gather - http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/jijoy-sep2-pr/jijoy-sep2-pr_20220902T041842/logs/testcases_1662115760/
=================================================================
Version-Release number of selected component (if applicable):
OCP 4.10.28
ODF 4.11.1-5
$ oc get csv
NAME                                      DISPLAY                       VERSION           REPLACES                                  PHASE
mcg-operator.v4.11.1                      NooBaa Operator               4.11.1            mcg-operator.v4.11.0                      Succeeded
ocs-operator.v4.11.1                      OpenShift Container Storage   4.11.1            ocs-operator.v4.11.0                      Succeeded
ocs-osd-deployer.v2.0.5                   OCS OSD Deployer              2.0.5             ocs-osd-deployer.v2.0.4                   Succeeded
odf-csi-addons-operator.v4.11.1           CSI Addons                    4.11.1            odf-csi-addons-operator.v4.11.0           Succeeded
odf-operator.v4.11.1                      OpenShift Data Foundation     4.11.1            odf-operator.v4.11.0                      Succeeded
ose-prometheus-operator.4.10.0            Prometheus Operator           4.10.0            ose-prometheus-operator.4.8.0             Succeeded
route-monitor-operator.v0.1.422-151be96   Route Monitor Operator        0.1.422-151be96   route-monitor-operator.v0.1.420-b65f47e   Succeeded


==========================================================
How reproducible:
Reporting the first occurrence

Steps to Reproduce:
1. Upgrade provider and consumer from ODF 4.10.z to 4.11.1
(performed uninstallation of ODF 4.10.5 and installed ODF 4.11.1 to test this because upgrade process was not working) 
2. Check the status of report-status-to-provider pod in the consumer.

Actual results:
$ oc get pods | grep report-status-to-provider
report-status-to-provider-27701857-fzvsl                          0/1     CreateContainerError   0             75m
report-status-to-provider-27701858-vgcq7                          0/1     CreateContainerError   0             74m
report-status-to-provider-27701859-xnwc9                          0/1     CreateContainerError   0             73m
report-status-to-provider-27701860-4m6nh                          0/1     CreateContainerError   0             72m
report-status-to-provider-27701861-srmjj                          0/1     CreateContainerError   0             71m
report-status-to-provider-27701862-8n7ds                          0/1     CreateContainerError   0             70m
report-status-to-provider-27701863-xmbvn                          0/1     CreateContainerError   0             69m
report-status-to-provider-27701864-sfx7c                          0/1     CreateContainerError   0             68m
report-status-to-provider-27701865-fccd5                          0/1     CreateContainerError   0             67m
report-status-to-provider-27701866-spn5h                          0/1     CreateContainerError   0             66m
report-status-to-provider-27701867-4rrcz                          0/1     CreateContainerError   0             65m
report-status-to-provider-27701868-5h4z8                          0/1     CreateContainerError   0             64m
report-status-to-provider-27701869-5shhf                          0/1     CreateContainerError   0             63m
report-status-to-provider-27701870-8sxr8                          0/1     CreateContainerError   0             62m
report-status-to-provider-27701871-22rjf                          0/1     CreateContainerError   0             61m
report-status-to-provider-27701872-fgx6n                          0/1     CreateContainerError   0             60m
report-status-to-provider-27701873-vgf7k                          0/1     CreateContainerError   0             59m
report-status-to-provider-27701874-pdllq                          0/1     CreateContainerError   0             58m
report-status-to-provider-27701875-ddjwh                          0/1     CreateContainerError   0             57m
report-status-to-provider-27701876-mk5wh                          0/1     CreateContainerError   0             56m
report-status-to-provider-27701877-5twxv                          0/1     CreateContainerError   0             55m
report-status-to-provider-27701878-x7ggm                          0/1     CreateContainerError   0             54m
report-status-to-provider-27701879-7s7zf                          0/1     CreateContainerError   0             53m
report-status-to-provider-27701880-6gv96                          0/1     CreateContainerError   0             52m
report-status-to-provider-27701881-hrszq                          0/1     CreateContainerError   0             51m
report-status-to-provider-27701882-9bpgp                          0/1     CreateContainerError   0             50m
report-status-to-provider-27701883-dwmv4                          0/1     CreateContainerError   0             49m
report-status-to-provider-27701884-gv7ps                          0/1     CreateContainerError   0             48m
report-status-to-provider-27701885-j4p7p                          0/1     CreateContainerError   0             47m
report-status-to-provider-27701886-7dmv8                          0/1     CreateContainerError   0             46m
report-status-to-provider-27701887-dmcjb                          0/1     CreateContainerError   0             45m
report-status-to-provider-27701888-pxm42                          0/1     CreateContainerError   0             44m
report-status-to-provider-27701889-g4fc6                          0/1     CreateContainerError   0             43m
report-status-to-provider-27701890-t9hql                          0/1     CreateContainerError   0             42m
report-status-to-provider-27701891-lzcts                          0/1     CreateContainerError   0             41m
report-status-to-provider-27701892-x8qmc                          0/1     CreateContainerError   0             40m
report-status-to-provider-27701893-bjwwn                          0/1     CreateContainerError   0             39m
report-status-to-provider-27701894-k4vsh                          0/1     CreateContainerError   0             38m
report-status-to-provider-27701895-gvmq6                          0/1     CreateContainerError   0             37m
report-status-to-provider-27701896-qf7nh                          0/1     CreateContainerError   0             36m
report-status-to-provider-27701897-mwlhz                          0/1     CreateContainerError   0             35m
report-status-to-provider-27701898-v7qbn                          0/1     CreateContainerError   0             34m
report-status-to-provider-27701899-vcbfs                          0/1     CreateContainerError   0             33m
report-status-to-provider-27701900-c55dr                          0/1     CreateContainerError   0             32m
report-status-to-provider-27701901-4snlz                          0/1     CreateContainerError   0             31m
report-status-to-provider-27701902-t5n7l                          0/1     CreateContainerError   0             30m
report-status-to-provider-27701903-znhrd                          0/1     CreateContainerError   0             29m
report-status-to-provider-27701904-99djk                          0/1     CreateContainerError   0             28m
report-status-to-provider-27701905-97bw7                          0/1     CreateContainerError   0             27m
report-status-to-provider-27701906-6bcg4                          0/1     CreateContainerError   0             26m
report-status-to-provider-27701907-hgbxl                          0/1     CreateContainerError   0             25m
report-status-to-provider-27701908-spwzp                          0/1     CreateContainerError   0             24m
report-status-to-provider-27701909-dfd8x                          0/1     CreateContainerError   0             23m
report-status-to-provider-27701910-rrfmx                          0/1     CreateContainerError   0             22m
report-status-to-provider-27701911-mv5pn                          0/1     CreateContainerError   0             21m
report-status-to-provider-27701912-wskz6                          0/1     CreateContainerError   0             20m
report-status-to-provider-27701913-lcpfm                          0/1     CreateContainerError   0             19m
report-status-to-provider-27701914-7lc9c                          0/1     CreateContainerError   0             18m
report-status-to-provider-27701915-nbr6b                          0/1     CreateContainerError   0             17m
report-status-to-provider-27701916-hxwwm                          0/1     CreateContainerError   0             16m
report-status-to-provider-27701917-46wvw                          0/1     CreateContainerError   0             15m
report-status-to-provider-27701918-j9x4t                          0/1     CreateContainerError   0             14m
report-status-to-provider-27701919-cvwj9                          0/1     CreateContainerError   0             13m
report-status-to-provider-27701920-9c8ns                          0/1     CreateContainerError   0             12m
report-status-to-provider-27701921-t69zz                          0/1     CreateContainerError   0             11m
report-status-to-provider-27701922-j68d7                          0/1     CreateContainerError   0             10m
report-status-to-provider-27701923-5dvkj                          0/1     CreateContainerError   0             9m8s
report-status-to-provider-27701924-pbdrz                          0/1     CreateContainerError   0             8m8s
report-status-to-provider-27701925-x8tlj                          0/1     CreateContainerError   0             7m8s
report-status-to-provider-27701926-x89gk                          0/1     CreateContainerError   0             6m8s
report-status-to-provider-27701927-zt7cp                          0/1     CreateContainerError   0             5m8s
report-status-to-provider-27701928-k9z4q                          0/1     CreateContainerError   0             4m8s
report-status-to-provider-27701929-vhgx4                          0/1     CreateContainerError   0             3m8s
report-status-to-provider-27701930-bvd86                          0/1     CreateContainerError   0             2m8s
report-status-to-provider-27701931-tn4lj                          0/1     CreateContainerError   0             68s
report-status-to-provider-27701932-4ssgd                          0/1     CreateContainerError   0             8s


Expected results:
report-status-to-provider pod should not have any issue.

Additional info:

Comment 2 Dhruv Bindra 2022-09-06 13:01:41 UTC
After debugging Jilju's cluster, I got to know that status-reporter binary is missing from the container which resulted into status-reporter pod going into the CreateContainerError state. I have informed the build team about it and they have raised the MR to fix the build pipeline. MR: https://gitlab.cee.redhat.com/ceph/rhodf/-/merge_requests/689

Comment 4 krishnaram Karthick 2022-09-07 07:33:08 UTC
proposing the fix into 4.11.1 as this blocks verification of https://bugzilla.redhat.com/show_bug.cgi?id=2120314#c11

Comment 7 Dhruv Bindra 2022-09-12 07:51:10 UTC
*** Bug 2123724 has been marked as a duplicate of this bug. ***

Comment 8 Jilju Joy 2022-09-12 10:02:23 UTC
Verified in version:
ODF 4.11.1-8
OCP 4.10.30
ocs-osd-deployer.v2.0.5

Pods "report-status-to-provider" are in Completed state in the consumer cluster.
$ oc get pods -o wide | grep report-status-to-provider
report-status-to-provider-27716276-wn6fw           0/1     Completed   0             2m57s   10.129.2.49    ip-10-0-173-22.ec2.internal    <none>           <none>
report-status-to-provider-27716277-nrn4z           0/1     Completed   0             117s    10.129.2.50    ip-10-0-173-22.ec2.internal    <none>           <none>
report-status-to-provider-27716278-dvcpd           0/1     Completed   0             57s     10.131.0.46    ip-10-0-157-29.ec2.internal    <none>           <none>

Comment 13 errata-xmlrpc 2022-09-14 15:15:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenShift Data Foundation 4.11.1 Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:6525