Bug 1870439 - [deploy cnv] various Init:ImagePullBackOff when deploying CNV on indeximage format
Summary: [deploy cnv] various Init:ImagePullBackOff when deploying CNV on indeximage f...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Installation
Version: 2.4.1
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: ---
Assignee: Simone Tiraboschi
QA Contact: Inbar Rose
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-08-20 05:53 UTC by Tareq Alayan
Modified: 2020-08-20 15:33 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-08-20 15:33:22 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Tareq Alayan 2020-08-20 05:53:15 UTC
Description of problem:
kube-cni-linux-bridge-plugin-v8w5t
nmstate-handler-42zsg
ovs-cni-amd64-s26hh
are not ready due to Init:ImagePullBackOff

Version-Release number of selected component (if applicable):
registry-proxy.engineering.redhat.com/rh-osbs/iib:3059

How reproducible:
always

Steps to Reproduce:
1. deploy cnv
2.
3.


Additional info:


oc describe pod ovs-cni-amd64-s26hh -nopenshift-cnv
Events:
  Type     Reason   Age                      From                                Message
  ----     ------   ----                     ----                                -------
  Warning  Failed   164m (x78 over 6h53m)    kubelet, talayan241-ll2bx-master-2  (combined from similar events): Error: ImagePullBackOff
  Normal   Pulling  34m (x81 over 7h9m)      kubelet, talayan241-ll2bx-master-2  Pulling image "registry.redhat.io/container-native-virtualization/ovs-cni-plugin@sha256:026740eab93d30dc16fc2538ecda58a4341cde3b8722462eb7c9cea67b676cf1"
  Warning  Failed   9m14s (x1806 over 7h9m)  kubelet, talayan241-ll2bx-master-2  Error: ImagePullBackOff
  Normal   BackOff  4m10s (x1841 over 7h9m)  kubelet, talayan241-ll2bx-master-2  Back-off pulling image "registry.redhat.io/container-native-virtualization/ovs-cni-plugin@sha256:026740eab93d30dc16fc2538ecda58a4341cde3b8722462eb7c9cea67b676cf1"

oc describe pod nmstate-handler-42zsg -nopenshift-cnv
Normal   Pulling  37m (x81 over 7h13m)      kubelet, talayan241-ll2bx-master-2  Pulling image "registry.redhat.io/container-native-virtualization/kubernetes-nmstate-handler-rhel8@sha256:f17f2ef133df6e985118f3dc21a1d3ef67dc66bd68e7556af7bc2b0c4c68c4b8"
  Normal   BackOff  7m59s (x1841 over 7h12m)  kubelet, talayan241-ll2bx-master-2  Back-off pulling image "registry.redhat.io/container-native-virtualization/kubernetes-nmstate-handler-rhel8@sha256:f17f2ef133df6e985118f3dc21a1d3ef67dc66bd68e7556af7bc2b0c4c68c4b8"
  Warning  Failed   2m59s (x1831 over 7h12m)  kubelet, talayan241-ll2bx-master-2  Error: ImagePullBackOff

oc describe pod kube-cni-linux-bridge-plugin-v8w5t -nopenshift-cnv
 Warning  Failed   29m (x78 over 7h14m)     kubelet, talayan241-ll2bx-master-2  Error: ErrImagePull
  Normal   Pulling  18m (x85 over 7h14m)     kubelet, talayan241-ll2bx-master-2  Pulling image "registry.redhat.io/container-native-virtualization/cnv-containernetworking-plugins@sha256:91e7c75ec5db17e73eceb37a02cf0d26643724a66cb502f67d0cf3646a6b6308"
  Warning  Failed   9m (x1841 over 7h14m)    kubelet, talayan241-ll2bx-master-2  Error: ImagePullBackOff
  Normal   BackOff  4m8s (x1868 over 7h14m)  kubelet, talayan241-ll2bx-master-2  Back-off pulling image "registry.redhat.io/container-native-virtualization/cnv-containernetworking-plugins@sha256:91e7c75ec5db17e73eceb37a02cf0d26643724a66cb502f67d0cf3646a6b6308"

Comment 3 Simone Tiraboschi 2020-08-20 09:18:25 UTC
Tareq was using oc client 4.4.0-0.nightly-2020-02-17-022408 which had a bug on this area.
Using an up-to-date oc client to generate the ICSP should be enough to avoid this.

Comment 4 Tareq Alayan 2020-08-20 11:38:38 UTC
I was using 
./oc version --client
Client Version: 4.5.6


but now i see all images running:
oc get pods -nopenshift-cnv
NAME                                                  READY   STATUS    RESTARTS   AGE
bridge-marker-5t5gt                                   1/1     Running   0          19m
bridge-marker-6h6gh                                   1/1     Running   0          19m
bridge-marker-8sj7d                                   1/1     Running   0          19m
bridge-marker-cv4p5                                   1/1     Running   0          19m
bridge-marker-d7s6d                                   1/1     Running   0          19m
bridge-marker-qql2w                                   1/1     Running   0          19m
cdi-apiserver-7c4566c98c-2fw8f                        1/1     Running   0          14m
cdi-deployment-79fdcfdccb-xlprz                       1/1     Running   0          14m
cdi-operator-864b9f49f4-6lfxz                         1/1     Running   0          14m
cdi-uploadproxy-5d4cc54b4c-xr825                      1/1     Running   0          14m
cluster-network-addons-operator-54cdb99964-tkqgd      1/1     Running   0          14m
hco-operator-685f75b94b-22bmr                         1/1     Running   0          14m
hostpath-provisioner-h2gdw                            1/1     Running   0          7m59s
hostpath-provisioner-lj576                            1/1     Running   0          7m59s
hostpath-provisioner-operator-8446b654f4-5lmr6        1/1     Running   0          14m
hostpath-provisioner-x6k8h                            1/1     Running   0          7m59s
kube-cni-linux-bridge-plugin-dmvp5                    1/1     Running   0          19m
kube-cni-linux-bridge-plugin-fx7x8                    1/1     Running   0          19m
kube-cni-linux-bridge-plugin-qbhzd                    1/1     Running   0          19m
kube-cni-linux-bridge-plugin-tb74b                    1/1     Running   0          19m
kube-cni-linux-bridge-plugin-w425l                    1/1     Running   0          19m
kube-cni-linux-bridge-plugin-wrbf8                    1/1     Running   0          19m
kubemacpool-mac-controller-manager-6f9c447bbd-gh94j   1/1     Running   0          14m
kubevirt-node-labeller-d8ndj                          1/1     Running   0          18m
kubevirt-node-labeller-mh58s                          1/1     Running   0          18m
kubevirt-node-labeller-rkpb7                          1/1     Running   0          18m
kubevirt-ssp-operator-6c5cd84db9-dkj75                1/1     Running   0          14m
nmstate-handler-24jrf                                 1/1     Running   0          19m
nmstate-handler-2ljhn                                 1/1     Running   1          19m
nmstate-handler-bsx9t                                 1/1     Running   0          19m
nmstate-handler-r6gcp                                 1/1     Running   0          19m
nmstate-handler-rphsb                                 1/1     Running   1          19m
nmstate-handler-vwfwb                                 1/1     Running   0          19m
node-maintenance-operator-59c7b6c54b-gdbdh            1/1     Running   0          20m
ovs-cni-amd64-29nc7                                   1/1     Running   1          19m
ovs-cni-amd64-7mlxn                                   1/1     Running   1          19m
ovs-cni-amd64-7nm6m                                   1/1     Running   0          19m
ovs-cni-amd64-pvq58                                   1/1     Running   0          19m
ovs-cni-amd64-tln82                                   1/1     Running   0          19m
ovs-cni-amd64-z9w7s                                   1/1     Running   0          19m
virt-api-7686f978db-4g4s5                             1/1     Running   0          11m
virt-api-7686f978db-zzfpn                             1/1     Running   0          14m
virt-controller-7d567db8c6-w9j59                      1/1     Running   0          14m
virt-controller-7d567db8c6-wlf2h                      1/1     Running   0          11m
virt-handler-bxnc5                                    1/1     Running   0          18m
virt-handler-grqb7                                    1/1     Running   0          18m
virt-handler-k2fps                                    1/1     Running   0          18m
virt-operator-5ff4447fd-d8zft                         1/1     Running   0          11m
virt-operator-5ff4447fd-mkmxl                         1/1     Running   0          14m
virt-template-validator-5d9bbfbcc7-5s2sc              1/1     Running   0          14m
virt-template-validator-5d9bbfbcc7-l5j8k              1/1     Running   0          11m
vm-import-controller-58469cdfcf-65cfs                 1/1     Running   0          14m
vm-import-operator-77c867b45f-xbwmz                   1/1     Running   0          14m

Comment 5 Simone Tiraboschi 2020-08-20 15:33:22 UTC
Not reproducible with up to date oc client, closing as not a bug.


Note You need to log in before you can comment on or make changes to this bug.