Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: (In reply to David Zager from comment #6) > Denys Shchedrivyi mentioned the following issue with the > kubevirt-node-labeller on the cnv-devel list (Subj: [cnv] > hco-bundle-registry:v2.0.0-13) that I believe comes from the > kubevirt-ssp-operator: > > [root@dell-r640-010 ~]# oc describe pod -n kubevirt-hyperconverged > kubevirt-node-labeller-hzwxl > . > Warning Failed 1h (x4 over 1h) kubelet, > working-jww4k-worker-0-6dqp9 Failed to pull image > "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native- > virtualization/kvm-info-nfd-plugin:v0.4.0": rpc error: code = Unknown desc = > Error reading manifest v0.4.0 in > brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native- > virtualization/kvm-info-nfd-plugin: unknown: Not Found > Warning Failed 1h (x4 over 1h) kubelet, > working-jww4k-worker-0-6dqp9 Error: ErrImagePull > Warning Failed 5m (x331 over 1h) kubelet, > working-jww4k-worker-0-6dqp9 Error: ImagePullBackOff > Normal BackOff 44s (x353 over 1h) kubelet, > working-jww4k-worker-0-6dqp9 Back-off pulling image > "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native- > virtualization/kvm-info-nfd-plugin:v0.4.0" Version-Release number of selected component (if applicable): hco-2.0.0-11 How reproducible: Steps to Reproduce: 1. Deploy CNV on top of OCP 2. 3. Actual results: labeler should come up Expected results: can not pull image of kvm-info-nfd-plugin Additional info: From Denys's mail [root@dell-r640-010 ~]# oc get pod -n kubevirt-hyperconverged NAME READY STATUS RESTARTS AGE cdi-operator-c78d55bd7-nftsp 1/1 Running 0 68m cluster-network-addons-operator-66655bd69b-xsvtt 1/1 Running 0 68m hco-operator-5d875c67b8-w7qqz 1/1 Running 0 68m kubevirt-node-labeller-hzwxl 0/1 Init:ImagePullBackOff 0 65m kubevirt-node-labeller-nspgj 0/1 Init:ImagePullBackOff 0 65m kubevirt-ssp-operator-578cb75cbd-w84fv 1/1 Running 0 68m kubevirt-web-ui-operator-6db98f677f-c2zd4 1/1 Running 0 68m node-maintenance-operator-5b489b5556-dnf8t 1/1 Running 0 68m virt-api-5d9f66dc76-qk64k 1/1 Running 0 66m virt-api-5d9f66dc76-x95wk 1/1 Running 0 66m virt-controller-6f8746f697-95tcp 1/1 Running 0 65m virt-controller-6f8746f697-m8tw4 1/1 Running 0 65m virt-handler-5rwnn 1/1 Running 0 65m virt-handler-cjxgt 1/1 Running 0 65m virt-operator-7f59d8df84-9jg9w 1/1 Running 0 68m virt-operator-7f59d8df84-p6kt7 1/1 Running 0 68m virt-template-validator-8dcd6dccc-96mnt 0/1 ImagePullBackOff 0 65m [root@dell-r640-010 ~]# oc describe pod -n kubevirt-hyperconverged kubevirt-node-labeller-hzwxl . Warning Failed 1h (x4 over 1h) kubelet, working-jww4k-worker-0-6dqp9 Failed to pull image "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native-virtualization/kvm-info-nfd-plugin:v0.4.0": rpc error: code = Unknown desc = Error reading manifest v0.4.0 in brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native-virtualization/kvm-info-nfd-plugin: unknown: Not Found Warning Failed 1h (x4 over 1h) kubelet, working-jww4k-worker-0-6dqp9 Error: ErrImagePull Warning Failed 5m (x331 over 1h) kubelet, working-jww4k-worker-0-6dqp9 Error: ImagePullBackOff Normal BackOff 44s (x353 over 1h) kubelet, working-jww4k-worker-0-6dqp9 Back-off pulling image "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native-virtualization/kvm-info-nfd-plugin:v0.4.0" [root@dell-r640-010 ~]# oc describe pod -n kubevirt-hyperconverged virt-template-validator-8dcd6dccc-96mnt . Warning Failed 1h (x4 over 1h) kubelet, working-jww4k-worker-0-z6t7b Failed to pull image "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native-virtualization/kubevirt-template-validator:v0.4.8": rpc error: code = Unknown desc = Error reading manifest v0.4.8 in brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native-virtualization/kubevirt-template-validator: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p>The requested URL /pulp/docker/v2/redhat-container-native-virtualization-kubevirt-template-validator/manifests/1/v0.4.8 was not found on this server.</p>\n</body></html>\n" Warning Failed 1h (x4 over 1h) kubelet, working-jww4k-worker-0-z6t7b Error: ErrImagePull Warning Failed 33m (x217 over 1h) kubelet, working-jww4k-worker-0-z6t7b Error: ImagePullBackOff Normal BackOff 3m (x350 over 1h) kubelet, working-jww4k-worker-0-z6t7b Back-off pulling image "brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/container-native-virtualization/kubevirt-template-validator:v0.4.8"
Will be fixed by new kubevirt-ssp-operator build and https://github.com/kubevirt/hyperconverged-cluster-operator/pull/94
Add testBlocker because it prevent us to create VM: # oc create -f vm-cirros.yaml Error from server (InternalError): error when creating "vm-cirros.yaml": Internal error occurred: failed calling admission webhook "virt-template-admission.kubevirt.io": Post https://virt-template-validator.kubevirt-hyperconverged.svc:443/virtualmachine-template-validate?timeout=30s: no endpoints available for service "virt-template-validator"
Looks good with hco-bundle-registry:v2.0.0-14 $ oc get pods -n kubevirt-hyperconverged NAME READY STATUS RESTARTS AGE cdi-operator-5cf5f7df79-g7l75 1/1 Running 0 11m cluster-network-addons-operator-5fc94fc94-bkczt 1/1 Running 0 11m hco-operator-64c9bc495d-gkvft 1/1 Running 0 11m kubevirt-node-labeller-hs6vm 1/1 Running 0 8m48s kubevirt-node-labeller-nb5jw 1/1 Running 0 8m48s kubevirt-ssp-operator-76bd5b68c-p9fw4 1/1 Running 0 11m kubevirt-web-ui-operator-5599867479-x5ndq 1/1 Running 0 11m node-maintenance-operator-7c4bd6546d-69qzl 1/1 Running 0 11m virt-api-5759c89548-4qrxh 1/1 Running 0 8m43s virt-api-5759c89548-bkns2 1/1 Running 0 8m43s virt-controller-75d85dc4d-mgtp5 1/1 Running 0 7m13s virt-controller-75d85dc4d-ncm2s 1/1 Running 0 7m13s virt-handler-jfgsc 1/1 Running 1 7m13s virt-handler-n2l9f 1/1 Running 0 7m13s virt-operator-74698f4dcd-27cjn 1/1 Running 0 11m virt-operator-74698f4dcd-dvspk 1/1 Running 0 11m virt-template-validator-85d7964c78-bgvlf 1/1 Running 0 8m48s
Has this been verified?
Verfied, see BZ 1709677#c12