Bug 1712455 - QE unable to test signed 4.1 non-CI builds until GA
Summary: QE unable to test signed 4.1 non-CI builds until GA
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Release
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.1.0
Assignee: Tim Bielawa
QA Contact: Mike Fiedler
URL:
Whiteboard: 4.1.2
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-21 14:36 UTC by Mike Fiedler
Modified: 2019-06-19 06:45 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-19 06:45:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-update-keys pull 11 0 None closed Concatenate keys into a single armored block 2020-10-05 00:40:04 UTC
Github openshift cluster-update-keys pull 12 0 None closed Only include the beta key. Not the beta and the release2 key 2020-10-05 00:40:04 UTC
Red Hat Product Errata RHBA-2019:1382 0 None None None 2019-06-19 06:45:44 UTC

Comment 1 Clayton Coleman 2019-05-21 14:59:47 UTC
Signing is not yet enabled on CI.

Currently blocker is that we need to get the SRE hosted signer up.

Comment 2 Clayton Coleman 2019-05-21 15:01:08 UTC
I ran the signer again, all current releases should be signed by 11:05am

Comment 3 Clayton Coleman 2019-05-21 15:01:51 UTC
Also, this has nothing to do with GA signing. 

Nightlies will not be signed.  We will only sign GA content.

Comment 4 Mike Fiedler 2019-05-21 15:29:07 UTC
Updating title

Comment 5 Clayton Coleman 2019-05-21 15:42:50 UTC
An option is for us to have ART sign rc.4 and rc.5.  We should be able to do that now.

Comment 7 Sudha Ponnaganti 2019-05-22 17:49:55 UTC
@mike - Can you use the latest RC to test upgrades. Looks like that seem to be working. Signing aspect is different as you already aware.

Comment 9 liujia 2019-05-23 08:12:56 UTC
Have a try on path 4.1.0-rc.4 to 4.1.0-rc.5, still need "--force".

scenario1: without "--force",fail
[root@preserve-jliu-worker 20190523_15261]# ./oc adm upgrade --to=4.1.0-rc.5
Updating to 4.1.0-rc.5
[root@preserve-jliu-worker 20190523_15261]# ./oc adm upgrade

info: An upgrade is in progress. Unable to apply 4.1.0-rc.5: the image may not be safe to use

Updates:

VERSION    IMAGE
4.1.0-rc.5 registry.svc.ci.openshift.org/ocp/release@sha256:dc67ad5edd91ca48402309fe0629593e5ae3333435ef8d0bc52c2b62ca725021

scenario2: with "--force" succeed.
[root@preserve-jliu-worker 20190523_15261]# ./oc adm upgrade --to=4.1.0-rc.5 --force
Updating to 4.1.0-rc.5
[root@preserve-jliu-worker 20190523_15261]# ./oc adm upgrade
info: An upgrade is in progress. Working towards 4.1.0-rc.5: downloading update

Updates:

VERSION    IMAGE
4.1.0-rc.5 registry.svc.ci.openshift.org/ocp/release@sha256:dc67ad5edd91ca48402309fe0629593e5ae3333435ef8d0bc52c2b62ca725021
[root@preserve-jliu-worker 20190523_15261]# ./oc adm upgrade
Cluster version is 4.1.0-rc.5

Comment 12 Tim Bielawa 2019-05-30 23:29:00 UTC
Signatures are now posted!

https://mirror.openshift.com/pub/openshift-v4/signatures/openshift/release/

1dabe42b5c94841fd8736d8f3a80afeaf5f5ad3833cef8d304c419a97b0efbc3 is for rc.6
7e1e73c66702daa39223b3e6dd2cf5e15c057ef30c988256f55fae27448c3b01 is for rc.7


This is following the format prescribed in the update keys operator: https://github.com/openshift/cluster-update-keys/blob/master/manifests.rhel/0000_90_cluster-update-keys_configmap.yaml#L5 

There are now signed sha256sum.txt.sig files in the rc.6 and rc.7 client tool directories: https://mirror.openshift.com/pub/openshift-v4/clients/ocp/

You can verify these were signed with the 'beta2' key by downloading each file and then running 'gpg --verify <the_file>'. You will see a header similar to the following:

> gpg: Signature made Thu 30 May 2019 06:21:48 PM CDT using RSA key ID F21541EB

But with different dates depending on the artifact.

This matches the official published key id posted on the Red Hat Product Signing Keys page: https://access.redhat.com/security/team/key

> 4096R/938a80caf21541eb (2009-02-24):
> Red Hat, Inc. (beta key 2) <security>

Comment 15 Mike Fiedler 2019-05-31 00:17:03 UTC
Attempted an upgrade of an rc6 cluster installed before the signatures were posted (to simulate beta 6 users who installed some time ago) to rc7 after changes in comment 12 were made.   Still seeing "image may not be safe to use":

[root@ip-172-31-53-199 ~]# oc get clusterversion                                                                        
NAME      VERSION      AVAILABLE   PROGRESSING   SINCE   STATUS                      
version   4.1.0-rc.6   True        False         8h      Cluster version is 4.1.0-rc.6    


[root@ip-172-31-53-199 ~]# oc adm upgrade --to-image=quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7
Updating to release image quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7

[root@ip-172-31-53-199 ~]# oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version             True        True          10s     Unable to apply quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7: the image may not be safe to use

Comment 16 Mike Fiedler 2019-05-31 01:05:43 UTC
Installed a cluster after rc6 was signed and tried an upgrade to signed rc7 and the result was the same as comment 15:

[root@ip-172-31-53-199 ~]# oc adm upgrade --to-image=quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7
Updating to release image quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7


[root@ip-172-31-53-199 ~]# oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version             True        True          5s      Unable to apply quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7: the image may not be safe to use

Comment 17 Tim Bielawa 2019-05-31 12:50:11 UTC
openshift/cluster-update-keys/pull/11/12 contain at least part of the required fixes. They're attached to this BZ now. One has not been merged yet.

Comment 18 Mike Fiedler 2019-05-31 23:11:27 UTC
Verified using 4.1.0.rc7, 4.1.0.rc8 and 4.1.0.rc9.

- rc7 and rc9 are both publicly signed and are valid targets of upgrades and downgrades
- rc8 and rc9 have the right key in their internal configmap to use during upgrade with the public keys

The tested upgrade/downgrade paths that work without --force are:

upgrade rc8 -> rc9
downgrade rc9 or rc8 -> rc7

All of these worked using these builds.   Note that no other builds can upgrade to rc9 without --force.

Comment 20 Yadan Pei 2019-06-03 07:51:57 UTC
1) 4.1.0-rc.8 -> 4.1.0-rc.9 -> 4.1.0-rc.7: upgrade first then downgrade, the downgrade path 4.1.0-rc.9 -> 4.1.0-rc.7 doesn't work without '--force' option, it has errors
$ oc adm upgrade
info: An upgrade is in progress. Unable to apply registry.svc.ci.openshift.org/ocp/release:4.1.0-rc.7: the image may not be safe to use

Updates:

VERSION                           IMAGE
4.1.0-0.nightly-2019-05-31-174150 registry.svc.ci.openshift.org/ocp/release@sha256:ba48e4781a7c7327fd5a155744b467019f10ba1be0b2d3bd59726afb1f898295
4.1.0-0.nightly-2019-05-29-220142 registry.svc.ci.openshift.org/ocp/release@sha256:bcd33081bc69c146bcad9f31d8a236a03996822b8e052640ccf15d78666df759

2) 4.1.0-rc.9 -> 4.1.0-rc.8 doesn't work without '--force' option, it has error
$ oc adm upgrade
info: An upgrade is in progress. Unable to apply registry.svc.ci.openshift.org/ocp/release:4.1.0-rc.8: the image may not be safe to use

Updates:

VERSION                           IMAGE
4.1.0-0.nightly-2019-05-31-174150 registry.svc.ci.openshift.org/ocp/release@sha256:ba48e4781a7c7327fd5a155744b467019f10ba1be0b2d3bd59726afb1f898295
4.1.0-0.nightly-2019-05-29-220142 registry.svc.ci.openshift.org/ocp/release@sha256:bcd33081bc69c146bcad9f31d8a236a03996822b8e052640ccf15d78666df759

3) 4.1.0-rc.9 -> 4.1.0-rc.7 direct downgrade doesn't work without '--force' option, it has error
$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version             True        True          8m7s    Unable to apply registry.svc.ci.openshift.org/ocp/release:4.1.0-rc.7: the image may not be safe to use

$ oc version
Client Version: version.Info{Major:"4", Minor:"1+", GitVersion:"v4.1.0-201905191700+7bd2e5b-dirty", GitCommit:"7bd2e5b", GitTreeState:"dirty", BuildDate:"2019-05-19T23:52:43Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+838b4fa", GitCommit:"838b4fa", GitTreeState:"clean", BuildDate:"2019-05-19T23:51:04Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Comment 21 Sunil Choudhary 2019-06-03 09:14:46 UTC
Upgrade from rc.8 to rc.9 successful.

oc adm upgrade --to=4.1.0-rc.9

Upgrade not safe error from rc.9 to rc.7 if attempted with tag as below:

oc adm upgrade --to-image=quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7

Uppgrade successful from rc.9 to rc.7 with sha image id as below.

$ oc adm upgrade --to-image=quay.io/openshift-release-dev/ocp-release@sha256:7e1e73c66702daa39223b3e6dd2cf5e15c057ef30c988256f55fae27448c3b01
Updating to release image quay.io/openshift-release-dev/ocp-release@sha256:7e1e73c66702daa39223b3e6dd2cf5e15c057ef30c988256f55fae27448c3b01

$ oc get clusterversion
NAME      VERSION      AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.1.0-rc.7   True        False         5m38s   Cluster version is 4.1.0-rc.7

$ oc version
Client Version: version.Info{Major:"4", Minor:"1+", GitVersion:"v4.1.0-201905021432+7903b3e-dirty", GitCommit:"7903b3e", GitTreeState:"dirty", BuildDate:"2019-05-02T19:09:52Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.4+838b4fa", GitCommit:"838b4fa", GitTreeState:"clean", BuildDate:"2019-05-19T23:51:04Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Comment 22 Sunil Choudhary 2019-06-03 10:21:18 UTC
Same behaviour while upgrading from rc.8 to rc.7. With tag, it reports "image not safe. With image sha256 ID, it is successful.

$ oc adm upgrade --to-image=quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7
Updating to release image quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7

$ oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version             True        True          2s      Unable to apply quay.io/openshift-release-dev/ocp-release:4.1.0-rc.7: the image may not be safe to use


$ oc adm upgrade --to-image=quay.io/openshift-release-dev/ocp-release@sha256:7e1e73c66702daa39223b3e6dd2cf5e15c057ef30c988256f55fae27448c3b01
Updating to release image quay.io/openshift-release-dev/ocp-release@sha256:7e1e73c66702daa39223b3e6dd2cf5e15c057ef30c988256f55fae27448c3b01

$ oc get clusterversion
NAME      VERSION      AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.1.0-rc.7   True        False         27s     Cluster version is 4.1.0-rc.7

Comment 24 errata-xmlrpc 2019-06-19 06:45:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1382


Note You need to log in before you can comment on or make changes to this bug.