2215083 – ImagePullBackOff for cluster-network-addons-operator and hco-operator

Bug 2215083 - ImagePullBackOff for cluster-network-addons-operator and hco-operator [NEEDINFO]

Summary: ImagePullBackOff for cluster-network-addons-operator and hco-operator

Keywords:
Status:	NEW
Alias:	None
Product:	Container Native Virtualization (CNV)
Classification:	Red Hat
Component:	Installation
Sub Component:
Version:	4.10.9
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	urgent
Target Milestone:	---
Target Release:	---
Assignee:	Simone Tiraboschi
QA Contact:	Natalie Gavrielov
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2023-06-14 17:10 UTC by Alberto Gonzalez
Modified:	2023-08-10 14:23 UTC (History)
CC List:	2 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:
Embargoed:
Dependent Products:
Flags:	stirabos: needinfo? (alberto.gonzalez)

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Issue Tracker	CNV-29918	0	None	None	None	2023-06-14 17:11:35 UTC

Description Alberto Gonzalez 2023-06-14 17:10:47 UTC

Description of problem:

Images for version 4.10.9 are rebased and now the digest is not the correct. Pulling the images for cluster-network-addons-operator and hco-operator fails


Version-Release number of selected component (if applicable):
4.10.9


How reproducible:
Install CNV 4.10.9


Actual results:

Back-off pulling image "registry.redhat.io/container-native-virtualization/cluster-network-addons-operator@sha256:74433260a14cc51099f875e6b2de763cb9a61661596bbd899d902be93c90a524"

Back-off pulling image "registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator@sha256:d6672501dc0ae1661a89db70b2f400b4fb5025c5724f937c2fd9c90d0d9c8cc0"


Expected results:

Pods running properly


Additional info:
[lab-user@provision ~]$ oc describe PackageManifest kubevirt-hyperconverged|grep -e addons-operator -e hyperconverged-cluster-operator
        Container Image:                                  registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator@sha256:d6672501dc0ae1661a89db70b2f400b4fb5025c5724f937c2fd9c90d0d9c8cc0
        Repository:                                       https://github.com/kubevirt/hyperconverged-cluster-operator
        registry.redhat.io/container-native-virtualization/cluster-network-addons-operator@sha256:74433260a14cc51099f875e6b2de763cb9a61661596bbd899d902be93c90a524
        registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator@sha256:d6672501dc0ae1661a89db70b2f400b4fb5025c5724f937c2fd9c90d0d9c8cc0


As you can see diesg images are different from yesterday:
https://catalog.redhat.com/software/containers/container-native-virtualization/hyperconverged-cluster-operator/5cac6a51d70cc57c44aaa9fe?tag=v4.10.9-4&push_date=1686670676000

https://catalog.redhat.com/software/containers/container-native-virtualization/cluster-network-addons-operator/5c8a92285a13464733ed08af?tag=v4.10.9-4&push_date=1686670676000

Comment 1 Simone Tiraboschi 2023-08-10 14:23:27 UTC

Today everything works as expected for me: those 2 images are correctly pullable.

Is this still relevant?


stirabos@t14s:~$ skopeo inspect docker://registry.redhat.io/container-native-virtualization/cluster-network-addons-operator@sha256:74433260a14cc51099f875e6b2de763cb9a61661596bbd899d902be93c90a524
{
    "Name": "registry.redhat.io/container-native-virtualization/cluster-network-addons-operator",
    "Digest": "sha256:74433260a14cc51099f875e6b2de763cb9a61661596bbd899d902be93c90a524",
    "RepoTags": [
        "v4.8.2",
        "v4.8.5-4",
        "v2.6.0-17",
        "v2.4.7",
        "v2.5.1-1",
        "v4.8.2-1",
        "v2.4.7-4",
        "v2.4.7-5",
        "v2.4.3",
        "v2.4.2",
        "v4.8.6-6-source",
        "v2.6.9-6",
        "v2.5.7",
        "v4.8.4",
        "v2.4.1",
        "v4.8.3",
        "v4.8.0",
        "v4.10.0-49",
        "v2.4.0",
        "v4.8.1",
        "v2.4.5",
        "v4.8.6-6",
        "v4.8.6",
        "v2.1.0",
        "v2.4.6-1",
        "v2.4.2-1",
        "v2.4.6",
        "v4.9.3-3",
        "v2.5.1",
        "v2.1.0-13",
        "v2.4.4",
        "v2.4.0-38",
        "v4.10.0",
        "v2.6.6",
        "v2.5.6",
        "v4.8.3-8",
        "v4.9.5-5-source",
        "v2.6.7-3",
        "v2.4.9",
        "v2.5.3",
        "v4.9.5",
        "v4.10.0-49-source",
        "v2.6.10-5-source",
        "v4.10.1-12-source",
        "v4.10.2-3",
        "v2.3.0-28",
        "v4.10.1",
        "2.4",
        "v2.5.5",
        "v4.8.1-2",
        "v2.2.0-8",
        "v4.9.4-7-source",
        "2.6",
        "v2.4.4-1",
        "v2.5.2",
        "v2.6.10",
        "v4.9.3-3-source",
        "v2.6.0",
        "v2.5.8",
        "v2.2.0",
        "v2.6.8-1",
        "v4.8.4-3",
        "v2.6.10-5",
        "v2.5.6-3",
        "v2.6.5",
        "v4.9.0-27",
        "v4.9.1",
        "v4.10.2",
        "v2.5.4-4",
        "v2.6.5-2",
        "v4.9.1-9-source",
        "v2.6.3",
        "v2.6.3-1",
        "v2.0.0-16",
        "v2.4.9-1",
        "v4.10.4",
        "v4.9.5-5",
        "v4.8.5",
        "v4.9.4-7",
        "v2.6.1-2",
        "v2.5.0",
        "v2.6.1",
        "v2.5.2-4",
        "v4.9.0",
        "v4.9.4",
        "v4.9.2",
        "v4.10.2-3-source",
        "v4.10.3",
        "v2.5.4",
        "v2.0.0",
        "v4.9.2-1",
        "v2.6.9",
        "v2.5.0-17",
        "v4.10.1-12",
        "v2.3.0",
        "v4.10.3-4",
        "v4.8.4-3-source",
        "v4.8.0-28",
        "v2.5.8-2",
        "v4.10.4-2",
        "v2.5.7-1",
        "v2.6.9-6-source",
        "v4.10.4-2-source",
        "v4.9.1-9",
        "v4.9.3",
        "v2.6.2",
        "v2.6.8",
        "v4.8.5-4-source",
        "v2.6.2-1",
        "v2.4.1-2",
        "v2.4.3-2",
        "v4.10.3-4-source",
        "v2.6.7",
        "v4.9.2-1-source",
        "v2.5.5-2",
        "v2.6.6-4",
        "v2.4.5-1",
        "v2.5.3-3",
        "v4.10.5",
        "v4.10.5-1",
        "v4.10.5-1-source",
        "v4.11.0-26-source",
        "v4.9.6-5-source",
        "v4.9.7-4-source",
        "v4.10.6-3-source",
        "v4.8.7",
        "v4.11.1-8",
        "v4.10.6",
        "v4.9.6",
        "v4.11.1-8-source",
        "v4.10.7-3",
        "v4.8.7-5-source",
        "v4.11.0-26",
        "v4.9.7-4",
        "v4.9",
        "v4.9.7",
        "v4.11.1",
        "v4.11.0",
        "v4.10.7",
        "v4.10.7-3-source",
        "v4.9.6-5",
        "v4.10.6-3",
        "v4.8.7-5",
        "v4.11.3-3-source",
        "v4.10.8",
        "v4.11.2-1-source",
        "v4.10.8-8",
        "v4.11.3-3",
        "v4.12.0",
        "v4.11.3",
        "v4.11.2",
        "v4.11.2-1",
        "v4.12.2-2",
        "v4.12.2-2-source",
        "v4.12.0-24",
        "v4.12.2",
        "v4.12.1",
        "v4.12.1-4",
        "v4.12.0-24-source",
        "v4.12.1-4-source",
        "v4.10.8-8-source",
        "v4.11.4-7",
        "v4.11.4",
        "v4.12.3",
        "v4.12.3-2",
        "v4.11.4-7-source",
        "v4.12.3-2-source",
        "v4.10.9",
        "v4.10.9-4",
        "v4.10.9-4-source",
        "v4.12.4-6",
        "v4.12.4",
        "v4.12.4-6-source",
        "v4.10.10-4",
        "v4.10",
        "v4.10.10",
        "v4.10.10-4-source",
        "v4.11.5-4",
        "v4.11.5",
        "v4.11",
        "v4.11.5-4-source",
        "v4.12.5-7",
        "v4.12",
        "v4.12.5",
        "v4.12.5-7-source"
    ],
    "Created": "2023-04-25T22:06:49.036615675Z",
    "DockerVersion": "",
    "Labels": {
        "architecture": "x86_64",
        "build-date": "2023-04-25T22:01:42",
        "com.redhat.component": "cluster-network-addons-operator-container",
        "com.redhat.license_terms": "https://www.redhat.com/agreements",
        "description": "Cluster Network Addons Operator",
        "distribution-scope": "public",
        "io.buildah.version": "1.27.3",
        "io.k8s.description": "Cluster Network Addons Operator",
        "io.k8s.display-name": "cluster-network-addons-operator",
        "io.openshift.expose-services": "",
        "io.openshift.tags": "cnv,operator,network",
        "maintainer": "phoracek",
        "name": "container-native-virtualization/cluster-network-addons-operator",
        "release": "4",
        "summary": "Cluster Network Addons Operator",
        "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/container-native-virtualization/cluster-network-addons-operator/images/v4.10.9-4",
        "vcs-ref": "b7de991426d2f71e4276ab92f6aaa0b1babcfb8c",
        "vcs-type": "git",
        "vendor": "Red Hat, Inc.",
        "version": "v4.10.9"
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Layers": [
        "sha256:7d8746ab4ad8e51637d5a71d17e876a7432c1bb34ba3db89985a0cb9500cd64b",
        "sha256:301090ddad26866fd2cf99e9290b9e652cc2f04b1ea10df969202f21dd8986c2"
    ],
    "LayersData": [
        {
            "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "Digest": "sha256:7d8746ab4ad8e51637d5a71d17e876a7432c1bb34ba3db89985a0cb9500cd64b",
            "Size": 82259072,
            "Annotations": null
        },
        {
            "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "Digest": "sha256:301090ddad26866fd2cf99e9290b9e652cc2f04b1ea10df969202f21dd8986c2",
            "Size": 39309638,
            "Annotations": null
        }
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "container=oci",
        "ENTRYPOINT=/entrypoint",
        "OPERATOR=/cluster-network-addons-operator",
        "MANIFEST_TEMPLATOR=/manifest-templator",
        "CSV_TEMPLATE=/cluster-network-addons-operator.VERSION.clusterserviceversion.yaml.in",
        "USER_UID=1001",
        "USER_NAME=cluster-network-addons-operator",
        "SOURCE_DIR=/remote-source/app"
    ]
}
stirabos@t14s:~$ skopeo inspect docker://registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator@sha256:d6672501dc0ae1661a89db70b2f400b4fb5025c5724f937c2fd9c90d0d9c8cc0
{
    "Name": "registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator",
    "Digest": "sha256:d6672501dc0ae1661a89db70b2f400b4fb5025c5724f937c2fd9c90d0d9c8cc0",
    "RepoTags": [
        "v2.5.3-3",
        "v4.8.1-3",
        "v2.6.9",
        "v4.8.5-6",
        "v4.8.6",
        "v2.4.4",
        "v2.6.9-4",
        "v2.5.0-53",
        "v4.9.2-6",
        "v2.4.4-1",
        "v2.5.1-2",
        "v2.6.5",
        "v2.4.9",
        "v2.4.9-2",
        "v2.5.1",
        "v4.10.2-2-source",
        "v4.10.3",
        "v4.8.2-2",
        "v2.6.10",
        "v2.6.1-7",
        "v4.9.3-4",
        "v2.5.5-3",
        "v4.8.1",
        "v2.4.2-1",
        "v2.4.8",
        "v4.9.4-5",
        "v2.0.0-32",
        "v4.10.4-2-source",
        "v4.10.1",
        "v2.2.0",
        "v4.10.2",
        "v4.10.0-133",
        "v2.5.4",
        "v2.0.0",
        "v2.5.2-3",
        "v2.4.5",
        "v2.4.0",
        "v2.6.7-1",
        "v2.5.0",
        "v4.10.2-2",
        "v4.9.2",
        "v2.6.5-1",
        "v2.6.2",
        "v2.1.0",
        "v2.6.6-4",
        "v2.5.5",
        "v2.4.1",
        "v2.5.7",
        "v2.4.0-70",
        "v2.6.3-3",
        "v4.8.3-5",
        "v2.6.8",
        "v2.6.10-4-source",
        "v4.9.1",
        "v4.8.5",
        "v4.9.5-6-source",
        "v2.5.6-5",
        "v2.3.0-61",
        "v4.10.3-4",
        "v2.5.2",
        "v2.4.6-2",
        "v2.3.0",
        "v4.8.6-3",
        "v2.6.0-77",
        "v4.9.5-6",
        "v2.4.2",
        "v4.8.4-4",
        "v2.5.4-6",
        "v4.8.2",
        "v2.6.8-1",
        "v2.4.7",
        "v4.8.5-6-source",
        "v4.9.1-6-source",
        "v4.8.3",
        "v4.9.2-6-source",
        "v4.8.0-62",
        "v2.4.8-3",
        "v2.6.6",
        "v2.4.3-2",
        "v4.8.0",
        "v4.8.4-4-source",
        "v4.10.3-4-source",
        "v2.5.8-2",
        "v4.10.0",
        "v2.6.9-4-source",
        "2.6",
        "v4.10.4-2",
        "v4.9.4-5-source",
        "v4.10.0-133-source",
        "v4.9.5",
        "v4.9.0",
        "v4.8.6-3-source",
        "v4.10.1-19-source",
        "v4.9.3",
        "v2.1.0-22",
        "v2.5.7-2",
        "v2.6.3",
        "v2.6.1",
        "v4.8.4",
        "v2.6.7",
        "v2.4.1-5",
        "v4.9.3-4-source",
        "v2.6.2-2",
        "v2.6.0",
        "v2.4.7-6",
        "v2.5.8",
        "v4.10.4",
        "v4.9.4",
        "v4.10.1-19",
        "v2.2.0-12",
        "v4.9.0-59",
        "v2.4.3",
        "v2.4.5-1",
        "v2.5.3",
        "v4.9.1-6",
        "v2.5.6",
        "v2.6.10-4",
        "2.4",
        "v2.4.6",
        "v4.10.5",
        "v4.10.5-1",
        "v4.10.5-1-source",
        "v4.8.7",
        "v4.9.6",
        "v4.9",
        "v4.9.6-3",
        "v4.9.7",
        "v4.9.7-4",
        "v4.10.7-2",
        "v4.8.7-4",
        "v4.11.1-10",
        "v4.11.1",
        "v4.11.0",
        "v4.10.7",
        "v4.10.6",
        "v4.11.0-96",
        "v4.9.6-3-source",
        "v4.9.7-4-source",
        "v4.10.6-4-source",
        "v4.11.1-10-source",
        "v4.10.6-4",
        "v4.10.7-2-source",
        "v4.8.7-4-source",
        "v4.11.0-96-source",
        "v4.12.2-2-source",
        "v4.12.0-96",
        "v4.10.8",
        "v4.12.1-5",
        "v4.12.1-5-source",
        "v4.11.2-2",
        "v4.12.0-96-source",
        "v4.11.2-2-source",
        "v4.10.8-2-source",
        "v4.11.2",
        "v4.10.8-2",
        "v4.12.1",
        "v4.11.3-2-source",
        "v4.11.3",
        "v4.11.3-2",
        "v4.12.2-2",
        "v4.12.2",
        "v4.12.0",
        "v4.11.4",
        "v4.11.4-7",
        "v4.12.3",
        "v4.12.3-4",
        "v4.12.3-4-source",
        "v4.11.4-7-source",
        "v4.10.9",
        "v4.10.9-4",
        "v4.10.9-4-source",
        "v4.12.4",
        "v4.12.4-7",
        "v4.12.4-7-source",
        "v4.10",
        "v4.10.10-3",
        "v4.10.10",
        "v4.10.10-3-source",
        "v4.11.5",
        "v4.11.5-4",
        "v4.11",
        "v4.11.5-4-source",
        "v4.12.5-7",
        "v4.12.5",
        "v4.12",
        "v4.12.5-7-source"
    ],
    "Created": "2023-04-26T07:48:07.2219238Z",
    "DockerVersion": "",
    "Labels": {
        "architecture": "x86_64",
        "build-date": "2023-04-26T07:42:53",
        "com.redhat.component": "hyperconverged-cluster-operator-container",
        "com.redhat.license_terms": "https://www.redhat.com/agreements",
        "description": "OpenShift Virtualization HyperConverged Cluster Operator",
        "distribution-scope": "public",
        "io.buildah.version": "1.27.3",
        "io.k8s.description": "OpenShift Virtualization HyperConverged Cluster Operator",
        "io.k8s.display-name": "hyperconverged-cluster-operator",
        "io.openshift.expose-services": "",
        "io.openshift.tags": "virtualization,kubevirt,operator",
        "maintainer": "Simone Tiraboschi \u003cstirabos\u003e",
        "name": "container-native-virtualization/hyperconverged-cluster-operator",
        "release": "4",
        "summary": "OpenShift Virtualization HyperConverged Cluster Operator",
        "upstream-vcs-ref": "dd1ccbbda5524959fa2bbf5e79f24b9841b79880",
        "upstream-vcs-type": "git",
        "upstream-version": "v1.6.0-30-gdd1ccbbd",
        "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/container-native-virtualization/hyperconverged-cluster-operator/images/v4.10.9-4",
        "vcs-ref": "c0acb9bf1f6c40b1dbbe9a00b35d8ecc1f361268",
        "vcs-type": "git",
        "vendor": "Red Hat, Inc.",
        "version": "v4.10.9"
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Layers": [
        "sha256:7d8746ab4ad8e51637d5a71d17e876a7432c1bb34ba3db89985a0cb9500cd64b",
        "sha256:159936233e356f70eb334eacd47f63f6abf7d21fd0b7d1582400b0e7a8ff250c"
    ],
    "LayersData": [
        {
            "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "Digest": "sha256:7d8746ab4ad8e51637d5a71d17e876a7432c1bb34ba3db89985a0cb9500cd64b",
            "Size": 82259072,
            "Annotations": null
        },
        {
            "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
            "Digest": "sha256:159936233e356f70eb334eacd47f63f6abf7d21fd0b7d1582400b0e7a8ff250c",
            "Size": 53649933,
            "Annotations": null
        }
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "container=oci",
        "USER_UID=1001",
        "USER_NAME=hyperconverged-cluster-operator",
        "KUBEVIRT_CLIENT_GO_SCHEME_REGISTRATION_VERSION=v1"
    ]
}

Note You need to log in before you can comment on or make changes to this bug.