Bug 1837675

Summary:	"oc adm release mirror" should create the configmap yaml for image signatures for use by restricted-network clusters
Product:	OpenShift Container Platform	Reporter:	Scott Dodson <sdodson>
Component:	oc	Assignee:	Jack Ottofaro <jack.ottofaro>
Status:	CLOSED ERRATA	QA Contact:	Johnny Liu <jialiu>
Severity:	high	Docs Contact:
Priority:	high
Version:	4.4	CC:	aos-bugs, jack.ottofaro, jialiu, jokerman, lmohanty, maszulik, mfojtik, wking, yinzhou
Target Milestone:	---	Keywords:	Upgrades
Target Release:	4.4.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:	Feature: Extends `oc adm release mirror ...` command to also create and apply ConfigMap manifests containing the release image signature which the cluster-version operator can use to verify the mirrored release. Reason: Currently a cluster upgrade can be accomplished on a cluster that does not have an active connection to the internet. However manual steps are required to create a ConfigMap containing the signature data required for update image verification. Result: This enhancement will automatically create the ConfigMap, so the user doesn't have to think about manual steps.	Story Points:	---
Clone Of:	1837404	Environment:
Last Closed:	2020-06-17 22:26:36 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1837404
Bug Blocks:

Description Scott Dodson 2020-05-19 19:12:00 UTC

+++ This bug was initially created as a clone of Bug #1837404 +++

Description of problem:

We have implemented [2] an enhancement request [1] which creates the configMap manifests containing the release image signature which the cluster-version operator could use to verify the mirrored release for disconnected clusters. 

We have a telecommunication service provider customer is going to be using disconnected OpenShift 4.4 clusters. 

Knowing this we had a discussion about what if anything we could/should backport pre-emptively to short circuit future pain, and avoid introducing bad habits like the use of --force as they do in 4.3 due to the lack of signature mirroring.

Hence we need to backport [2] to 4.4.z

[1] https://github.com/openshift/enhancements/pull/283/
[2] https://github.com/openshift/oc/pull/343 and the blocker bug fix PR which fixes the bug which was introduced in previous PR i.e. https://github.com/openshift/oc/pull/392

--- Additional comment from W. Trevor King on 2020-05-19 11:34:47 EDT ---

Bug 1825565's verification [1] was just about "is the old usage still broken?", not about "does the new feature work?".  I think we need this feature series to be a completely separate chain of bugs that motivate the feature backport.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1825565#c6

--- Additional comment from Scott Dodson on 2020-05-19 15:11:32 EDT ---

We'll just use this to verify the functionality of https://issues.redhat.com/browse/OTA-121 so that it may be backported.

Comment 1 Maciej Szulik 2020-05-20 15:44:26 UTC

PRs in the queue.

Comment 2 Jack Ottofaro 2020-05-20 18:11:37 UTC

We're planning on delivering this backport this sprint and I'm therefore removing keyword UpcomingSprint.

Comment 5 Johnny Liu 2020-06-02 15:54:13 UTC

Verified this bug with 4.4.0-0.nightly-2020-06-01-021027 (upgrade from 4.3.19), and PASS.


In this verification, I did the following scenario testing:
1. mirror release image to private registry and applying signature configmap directly to the target cluster, then update the air-gapped cluster without --force option, succeed.
$ oc adm release mirror --from=registry.svc.ci.openshift.org/ocp/release:4.4.0-0.nightly-2020-06-01-021027 --to=ec2-18-223-205-13.us-east-2.compute.amazonaws.com:5000/installer_autotest/ocp --to-release-image=ec2-18-223-205-13.us-east-2.compute.amazonaws.com:5000/installer_autotest/ocp:4.4.0-0.nightly-2020-06-01-021027 --apply-release-image-signature --overwrite
<--snip-->
configmap/sha256-6d02e83e08c9ff183f9e6197117bdd3ff6598d50357b7a66eddf003b0b59f815 created

$ oc get cm -n openshift-config-managed | grep '6d02e83e08c9ff183f9e6197117bdd3ff6598d50357b7a66eddf003b0b59f815'
sha256-6d02e83e08c9ff183f9e6197117bdd3ff6598d50357b7a66eddf003b0b59f815   1      2s

2. mirror release image to private registry, and saving signature configmap to a separate directoy, and apply it to the target cluster, then update the air-gapped cluster without --force option, succeed.
$ oc adm release mirror --from=registry.svc.ci.openshift.org/ocp/release:4.4.0-0.nightly-2020-06-01-021027 --to=auto-jialiu-652518.mirror-registry.qe.gcp.devcluster.openshift.com:5000/ocp/release --to-release-image=auto-jialiu-652518.mirror-registry.qe.gcp.devcluster.openshift.com:5000/ocp/release:4.4.0-0.nightly-2020-06-01-021027 --release-image-signature-to-dir=/home/installer-auto/workspace/installer-auto-test@7/assets_dir/OCP-27986_122892
<--snip-->
To apply signature configmaps use 'oc apply' on files found in /home/installer-auto/workspace/installer-auto-test@7/assets_dir/OCP-27986_122892

Configmap signature file /home/installer-auto/workspace/installer-auto-test@7/assets_dir/OCP-27986_122892/signature-sha256-6d02e83e08c9ff18.yaml created
Mirroring completed in 4m1.58s (24.15MB/s)

$ oc apply -f /home/installer-auto/workspace/installer-auto-test@7/assets_dir/OCP-27986_122892/signature-* --overwrite=true

configmap/sha256-6d02e83e08c9ff183f9e6197117bdd3ff6598d50357b7a66eddf003b0b59f815 created

3. mirror release image to local disk together with signature configmap yaml file, move cross firewall, upload disk files to private registry, and apply it to the target cluster, then update the air-gapped cluster without --force option, succeed.
$ oc adm release mirror --from=registry.svc.ci.openshift.org/ocp/release:4.4.0-0.nightly-2020-06-01-021027 --to-dir=/tmp/OCP-30833-cFAdcj/OCP-30833_02070349/data --to=file://test
<--snip-->
To upload local images to a registry, run:

  oc image mirror --from-dir=/tmp/OCP-30833-cFAdcj/OCP-30833_02070349/data 'file://test:4.4.0-0.nightly-2020-06-01-021027*' REGISTRY/REPOSITORY

Mirroring completed in 1m52.29s (51.96MB/s)
Configmap signature file /tmp/OCP-30833-cFAdcj/OCP-30833_02070349/data/config/signature-sha256-6d02e83e08c9ff18.yaml created

<move crosee firewall>

$ /opt/mirror-to-disk/OCP-30833_02070349/oc image mirror --from-dir=/opt/mirror-to-disk/OCP-30833_02070349/data 'file://test:4.4.0-0.nightly-2020-06-01-021027*' upshift-nointernet.mirror-registry.qe.devcluster.openshift.com:5000/ocp/release
<--snip-->
sha256:5ee39dad64860f101337541711e49acc9dce77491ee7397243de414263567e22 upshift-nointernet.mirror-registry.qe.devcluster.openshift.com:5000/ocp/release:4.4.0-0.nightly-2020-06-01-021027-mdns-publisher
Mirroring completed in 1.46s (0B/s)

$ KUBECONFIG=/opt/mirror-to-disk/OCP-30833_02070349/kubeconfig oc apply -f /opt/mirror-to-disk/OCP-30833_02070349/data/config/signature-*.yaml --overwrite=true
configmap/sha256-6d02e83e08c9ff183f9e6197117bdd3ff6598d50357b7a66eddf003b0b59f815 configured

Comment 9 errata-xmlrpc 2020-06-17 22:26:36 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2445