Bug 2062917

Summary:	ImageVerificationFailed when upgrade ocp in disconnected env
Product:	OpenShift Container Platform	Reporter:	yliu1
Component:	Telco Edge	Assignee:	Angie Wang <angwang>
Telco Edge sub component:	ZTP	QA Contact:	yliu1
Status:	CLOSED ERRATA	Docs Contact:
Severity:	medium
Priority:	unspecified	CC:	angwang, keyoung
Version:	4.10
Target Milestone:	---
Target Release:	4.11.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-08-26 16:43:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2067306

Description yliu1 2022-03-10 22:06:35 UTC

Description of problem:
In a disconnected env, we have two options to start an ocp upgrade after ocp image is mirrored to disconnected registry:
1) use force option in clusterversion to skip upgrade path and image signature verification
2) use upgrade graph in disconnected env, and this option requires an additional image signature verification configmap to avoid using "force".  

Currently, if we want to use option2, user would need to apply the configmap to every spoke manually. 

Version-Release number of selected component (if applicable):
4.10

How reproducible:
100%

Steps to Reproduce:
1. In a disconnected env, mirror the image to disconnected registry, and attempt to start upgrade
2. check clusterversion on spoke
3.

Actual results:
upgrade cannot start due to imageverficationfailed
- lastTransitionTime: "2022-03-10T21:29:11Z"
      message: 'The update cannot be verified: context deadline exceeded'
      reason: ImageVerificationFailed
      status: "True"
      type: Failing
    - lastTransitionTime: "2022-03-10T21:03:25Z"
      message: 'Unable to apply 4.10.4: the image may not be safe to use'
      reason: ImageVerificationFailed
      status: "True"
      type: Progressing

Expected results:
upgrade started

Additional info:
workaround is to manually apply a configmap on spoke like below to verify the signature.

[kni@provisionhost-0-0 ~]$ cat sigconfigmap.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: signature-yp-4.10.4
  namespace: openshift-config-managed
  labels:
    release.openshift.io/verification-signatures: ""
binaryData:
  sha256-9f9c3aaca64f62af992bae5de1e984571c8b812f598b74c84dc630b064389fb7-1 : owGbwMvMwMEoOU9/4l9n2UDGtYzJSWLxRQW5xZnpukWphbrhaV4VST56SZl5SZru8dVKyUWZJZnJiTlKVgrVSpm5iempYFZKfnJ2apFubmJeZlpqcYluSmY6kAJKKRVnJBqZmllZplkmGycmJieamaSZGSWmWVoaJSWmmqakGqZaWpiYmhsmWyRZGBqlmVpaJJmbJFuYpCSbGRskGZiZGFtYpiWZK9XqKCiVVBaArFNKLMnPzUxWSM7PK0nMzEstUgC6Ni+xpLQoVQmoKjMlNa8ks6QS2WFFqWmpRal5yWDthaWJlXqZ+fr5Bal5xRmZaSVA6ZzUxOJU3ZTUMv385AIY38pEz9BAz0S3wsIs3sxEqRbkhvyCksz8PGgAJBelAt1SBDI0KDVFwSOxRMEfaGgwyFCFYKCjMvPSFRxLSzLygcFWqWCgZ6BnCDSmk0mUmZUBFKDwgOfYtpH/N5u22eMtlzSfljhNbnx29s/FFZIpQTNY49rEYn/OdNxT3aY+p/qT+/VgK7nnIpyRMntrlnC6vE+++HJ3Z6gfi1/ne6ljwl/04mp7pude3NZ4yn4B8+prt7jPbPW88Vr1U/MXxtYLu+y3us1hTxKpNHZTd1oZLbav9WZQSUQHR4de+6Lm6Xsn1Lib7OlrXdzB8NLD2mfX+tV3bxXsmdk6Zf3DyT7vBQsdjvak7pBlUfTQ+NVeN+d8/qGkoLpk+6w74R7JC2YvS/7R4lfjsfn/I63wwGSmcAmL50Ylj3xnT3g2ddkW0dJVU3iac04YXuGalH1wc98UUcPbvFcc0jV/x5nr8Hy/OulIxY3QfoWjsq3WyesnbZ1zm2Hvx8vfDkTJCx6vb+Bb+dntp9d5hRjLyeZ1rU8cYr/eese7SfvSxw98JSoxkb6h76ffTXLeyyYSb/lu6uP4pyq8u+J/Bitrh6V1rI5e8MnbvvZUWvahJRP2uxzTS/j68byHZXfB91PbI5ZI/Pe77y/XFcIoeNAube9b0yO3N3EqtHF9kXH6x2B+RsrY5/vi/d4uLJU37q0Jnj5He9lnpedHZn5+dGHbL91bv4v57ky6GFbLb+w10+Vyr9798gjPkG8nJCaYvtS2ancpWBb9LXHhoRvdqt4/OIUjDmUuW+Lot/y8mGcy28Z74UvYVR9I5Vgs+5Vk+z5xu3//xRNf3qwMyPzeCgA=

Comment 2 yliu1 2022-03-28 14:44:36 UTC

Mark as verified to unblock backport to 4.10 as we currently can't install 4.11 builds yet in pipeline.