Bug 2222955

Summary: upgrade from ODF 4.10.13 to ODF 4.11.9 is failing on IBM Power
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: narayanspg <ngowda>
Component: odf-operatorAssignee: Nitin Goyal <nigoyal>
Status: CLOSED NOTABUG QA Contact: Elad <ebenahar>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.11CC: bkunal, muagarwa, ngowda, ocs-bugs, odf-bz-bot
Target Milestone: ---   
Target Release: ---   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-07-17 09:37:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description narayanspg 2023-07-14 15:18:41 UTC
Description of problem (please be detailed as possible and provide log
snippests):
upgrade from ODF 4.10.13 to ODF 4.11.9 is failing on IBM Power with Imagepullbackoff error.

Version of all relevant components (if applicable):
[root@nara-odf411-syd05-bastion-0 ~]# oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.43   True        False         8h      Cluster version is 4.11.43
[root@nara-odf411-syd05-bastion-0 ~]#

ODF version : 4.11.9-2


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Yes

Is there any workaround available to the best of your knowledge?
NO

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?
Yes

Can this issue reproduce from the UI?
Yes

If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1. create OCP 4.10 cluster and deploy ODF 4.10.13
2. upgrade OCP to 4.11
3. create catalog source for ODF 4.11.9 and upgrade the subscription to 4.11.9


Actual results:
ODF upgrade is failing with two pods in imagepullback error

Expected results:
ODF upgrade should succeed

Additional info:

Comment 2 narayanspg 2023-07-14 15:33:41 UTC
[root@nara2-odf411-syd05-bastion-0 ~]# oc get csv -n openshift-storage
NAME                               DISPLAY                       VERSION   REPLACES                           PHASE
mcg-operator.v4.10.13              NooBaa Operator               4.10.13   mcg-operator.v4.10.12              Succeeded
ocs-operator.v4.10.13              OpenShift Container Storage   4.10.13   ocs-operator.v4.10.12              Succeeded
odf-csi-addons-operator.v4.10.13   CSI Addons                    4.10.13   odf-csi-addons-operator.v4.10.12   Succeeded
odf-operator.v4.10.13              OpenShift Data Foundation     4.10.13   odf-operator.v4.10.12              Replacing
odf-operator.v4.11.9               OpenShift Data Foundation     4.11.9    odf-operator.v4.10.13              Failed
[root@nara2-odf411-syd05-bastion-0 ~]#

#oc get pods
odf-console-5c5d759cb5-dkmvr                                      0/1     ImagePullBackOff   0          14m
odf-operator-controller-manager-5f6bf74796-fbbb6                  1/2     ImagePullBackOff   0          14m

root@nara2-odf411-syd05-bastion-0 ~]# oc describe pod odf-console-5c5d759cb5-dkmvr

Events:
  Type     Reason          Age                From               Message
  ----     ------          ----               ----               -------
  Normal   Scheduled       15m                default-scheduler  Successfully assigned openshift-storage/odf-console-5c5d759cb5-dkmvr to syd05-worker-2.nara2-odf411.ibm.com by syd05-master-0.nara2-odf411.ibm.com
  Normal   AddedInterface  15m                multus             Add eth0 [10.131.0.164/23] from openshift-sdn
  Normal   Pulling         13m (x4 over 15m)  kubelet            Pulling image "registry.redhat.io/odf4/odf-console-rhel8@sha256:59bec5fa0cd4102b0d6d7b75919f4a0376b7ab397ea82c4389685a23e37c37d4"
  Warning  Failed          13m (x4 over 15m)  kubelet            Failed to pull image "registry.redhat.io/odf4/odf-console-rhel8@sha256:59bec5fa0cd4102b0d6d7b75919f4a0376b7ab397ea82c4389685a23e37c37d4": rpc error: code = Unknown desc = reading manifest sha256:59bec5fa0cd4102b0d6d7b75919f4a0376b7ab397ea82c4389685a23e37c37d4 in registry.redhat.io/odf4/odf-console-rhel8: manifest unknown: manifest unknown
  Warning  Failed          13m (x4 over 15m)  kubelet            Error: ErrImagePull
  Warning  Failed          13m (x6 over 15m)  kubelet            Error: ImagePullBackOff
  Normal   BackOff         2s (x65 over 15m)  kubelet            Back-off pulling image "registry.redhat.io/odf4/odf-console-rhel8@sha256:59bec5fa0cd4102b0d6d7b75919f4a0376b7ab397ea82c4389685a23e37c37d4"


[root@nara2-odf411-syd05-bastion-0 ~]# oc describe pod odf-operator-controller-manager-5f6bf74796-fbbb6
Events:
  Type     Reason          Age                 From               Message
  ----     ------          ----                ----               -------
  Normal   Scheduled       15m                 default-scheduler  Successfully assigned openshift-storage/odf-operator-controller-manager-5f6bf74796-fbbb6 to syd05-worker-2.nara2-odf411.ibm.com by syd05-master-0.nara2-odf411.ibm.com
  Normal   AddedInterface  15m                 multus             Add eth0 [10.131.0.163/23] from openshift-sdn
  Normal   Pulling         15m                 kubelet            Pulling image "registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:247a74298ff310d6e1992e21041e034e46fabaa5230d8ed553db27352d4441b5"
  Normal   Pulled          15m                 kubelet            Successfully pulled image "registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:247a74298ff310d6e1992e21041e034e46fabaa5230d8ed553db27352d4441b5" in 4.408432605s
  Normal   Created         15m                 kubelet            Created container kube-rbac-proxy
  Normal   Started         15m                 kubelet            Started container kube-rbac-proxy
  Warning  Failed          14m (x3 over 15m)   kubelet            Error: ErrImagePull
  Warning  Failed          14m (x5 over 15m)   kubelet            Error: ImagePullBackOff
  Normal   Pulling         14m (x4 over 15m)   kubelet            Pulling image "registry.redhat.io/odf4/odf-rhel8-operator@sha256:2bd829ca62a6a92b70440e067da9931977e71f2dd0ff4a71b4fc8fc708b930e8"
  Warning  Failed          14m (x4 over 15m)   kubelet            Failed to pull image "registry.redhat.io/odf4/odf-rhel8-operator@sha256:2bd829ca62a6a92b70440e067da9931977e71f2dd0ff4a71b4fc8fc708b930e8": rpc error: code = Unknown desc = reading manifest sha256:2bd829ca62a6a92b70440e067da9931977e71f2dd0ff4a71b4fc8fc708b930e8 in registry.redhat.io/odf4/odf-rhel8-operator: manifest unknown: manifest unknown
  Normal   BackOff         33s (x63 over 15m)  kubelet            Back-off pulling image "registry.redhat.io/odf4/odf-rhel8-operator@sha256:2bd829ca62a6a92b70440e067da9931977e71f2dd0ff4a71b4fc8fc708b930e8"
[root@nara2-odf411-syd05-bastion-0 ~]#

Comment 4 narayanspg 2023-07-17 07:29:42 UTC
Hi Kunal we are using the quay.io in our catalog source for ODF operator. 

we tried both manual upgrade and also through the script both are giving same results.

1. create 4.10 version OCP cluster.
2. deploy ODF 4.10.13
3. upgrade OCP to 4.11
4. update catalog source for 4.11.9 for ODF operator
5. update channel in subscription for ODF operator to 4.11
6. The upgrade is failed due to odf-console and odf-operator-controller-manager pods are failing.

the cluster got deleted over weekend. if required we can setup.

mustgather logs will be shared soon.

Comment 6 narayanspg 2023-07-17 08:00:18 UTC
mustgather logs are placed at below location:

https://drive.google.com/file/d/1uM1lZajtfn6scSj3I_5EoQwkbAi7QswP/view?usp=sharing

Comment 9 narayanspg 2023-07-17 09:30:54 UTC
after updating the ImageContentSourcePolicy , the image pull issue is resolved and upgrade is working fine.

Thanks for the help. we can close this case.

Comment 10 narayanspg 2023-07-17 09:32:00 UTC
found this as well.

https://github.com/red-hat-storage/ocs-ci/pull/7836/files