Bug 1797624

Summary: Upgradeable=False should not block updates within a z stream, but should block 4.y -> 4.(y+1) bumps
Product: OpenShift Container Platform Reporter: David Eads <deads>
Component: Cluster Version OperatorAssignee: David Eads <deads>
Status: CLOSED ERRATA QA Contact: Fan Jia <jfan>
Severity: high Docs Contact:
Priority: high    
Version: 4.4CC: aos-bugs, chezhang, jfan, jokerman, shurley, wking
Target Milestone: ---Keywords: Reopened
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1820231 (view as bug list) Environment:
Last Closed: 2020-05-04 11:29:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1820231    

Description David Eads 2020-02-03 14:04:26 UTC
We agreed on an approach a month ago.

Comment 2 Fan Jia 2020-02-07 23:18:27 UTC
Service catalog is remained in OCP 4.4 , when service catalog is enabled , it is ok to upgrade Cluster from 4.3 to 4.4 and also worked from 4.4-x to 4.4-x. 
The function can't be tested during the OCP 4.4 , should be checked during the OCP 4.5 and be checked upgrade from OCP 4.4 to OCP 4.5.

Comment 4 W. Trevor King 2020-04-02 14:52:35 UTC
We still want this softening, independent of the service catalog issue (if this had just been a service-catalog issue, a generic CVO fix would have been a pretty big hack ;).

Comment 5 W. Trevor King 2020-04-02 14:54:57 UTC
Hmm, I think this is still linked from the errata (at least, the Errata is still formally linked from the bug), so moving straight back into ON_QA.  Test should be "if any operator sets Upgradeable=False (not sure what an easy trigger for that would be), you can still get upgrades between 4.4 releases without having to force anything".  And, under the same conditions, updating to a 4.5 release should block on the Upgradeable=False.

Comment 9 W. Trevor King 2020-04-07 17:24:49 UTC
Moving to ASSIGNED so this doesn't get swept back in by the Errata sweeper or anything, until we get a signed 4.5 release.

Comment 10 W. Trevor King 2020-04-07 17:31:31 UTC
Actually, wait, there are signed 4.5 nightlies already.  For example [1]:

  $ curl -s https://mirror.openshift.com/pub/openshift-v4/signatures/openshift/release/sha256=b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc/signature-1 | gpg --decrypt
  {"critical": {"image": {"docker-manifest-digest": "sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc"}, "type": "atomic container signature", "identity": {"docker-reference": "quay.io/openshift-release-dev/ocp-release-nightly:4.5.0-0.nightly-2020-04-05-042758-x86_64"}}}gpg: Signature made Sat 04 Apr 2020 11:48:34 PM PDT using RSA key ID F21541EB
  gpg: Good signature from "Red Hat, Inc. (beta key 2) <security>"
  gpg:                 aka "Mark Cox Internal RSA 4096 test key <mjc>"
  gpg: WARNING: This key is not certified with a trusted signature!
  gpg:          There is no indication that the signature belongs to the owner.
  Primary key fingerprint: B08B 659E E86A F623 BC90  E8DB 938A 80CA F215 41EB

You should be able to select that target with:

  $ oc adm --allow-explicit-upgrade --to-image quay.io/openshift-release-dev/ocp-release-nightly@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc

without using --force.

[1]: https://mirror.openshift.com/pub/openshift-v4/clients/ocp-dev-preview/4.5.0-0.nightly-2020-04-05-042758/

Comment 11 Fan Jia 2020-04-08 00:56:21 UTC
(In reply to W. Trevor King from comment #10)
> 
> You should be able to select that target with:
> 
>   $ oc adm --allow-explicit-upgrade --to-image
> quay.io/openshift-release-dev/ocp-release-nightly@sha256:
> b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc
> 
> without using --force.
> 
> [1]:
> https://mirror.openshift.com/pub/openshift-v4/clients/ocp-dev-preview/4.5.0-
> 0.nightly-2020-04-05-042758/

Ok, thank you , I will try.

Comment 12 Fan Jia 2020-04-08 06:18:54 UTC
test result:
upgrade from 4.4.0-0.nightly-2020-04-04-025830 to 4.5.0-0.nightly-2020-04-05-042758
1.enable service-catalog
2.upgrade the cluster with command:
oc adm upgrade --allow-explicit-upgrade --to-image quay.io/openshift-release-dev/ocp-release-nightly@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc
3. the status of clusterversion, the upgrade is blocked
$ oc get clusterversion 
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.4.0-0.nightly-2020-04-04-025830   True        True          23m     Unable to apply registry.svc.ci.openshift.org/ocp/release@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc: it may not be safe to apply this update

$ oc get clusterversion  -o yaml
apiVersion: v1
items:
- apiVersion: config.openshift.io/v1
  kind: ClusterVersion
  metadata:
    creationTimestamp: "2020-04-08T03:25:52Z"
    generation: 4
    name: version
    resourceVersion: "38140"
    selfLink: /apis/config.openshift.io/v1/clusterversions/version
    uid: eed2368a-3794-435d-999e-8f7326b6b650
  spec:
    channel: stable-4.4
    clusterID: 8fbf5684-36f6-4418-a91a-c12d73fb8ee8
    desiredUpdate:
      force: false
      image: registry.svc.ci.openshift.org/ocp/release@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc
      version: ""
    upstream: https://api.openshift.com/api/upgrades_info/v1/graph
  status:
    availableUpdates: null
    conditions:
    - lastTransitionTime: "2020-04-08T03:45:25Z"
      message: Done applying 4.4.0-0.nightly-2020-04-04-025830
      status: "True"
      type: Available
    - lastTransitionTime: "2020-04-08T04:25:25Z"
      message: |-
        Precondition "ClusterVersionUpgradeable" failed because of "ClusterOperatorsNotUpgradeable": Multiple cluster operators cannot be upgradeable:
        * Cluster operator service-catalog-apiserver cannot be upgraded: _Managed: Upgradeable: the apiserver is in a managed state, upgrades are not possible.
        * Cluster operator service-catalog-controller-manager cannot be upgraded: _Managed: Upgradeable: the controller manager is in a managed state, upgrades are not possible.
      reason: UpgradePreconditionCheckFailed
      status: "True"
      type: Failing
    - lastTransitionTime: "2020-04-08T04:25:10Z"
      message: 'Unable to apply registry.svc.ci.openshift.org/ocp/release@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc:
        it may not be safe to apply this update'
      reason: UpgradePreconditionCheckFailed
      status: "True"
      type: Progressing
    - lastTransitionTime: "2020-04-08T03:25:56Z"
      message: 'Unable to retrieve available updates: currently installed version
        4.4.0-0.nightly-2020-04-04-025830 not found in the "stable-4.4" channel'
      reason: VersionNotFound
      status: "False"
      type: RetrievedUpdates
    - lastTransitionTime: "2020-04-08T03:50:51Z"
      message: |-
        Multiple cluster operators cannot be upgradeable:
        * Cluster operator service-catalog-apiserver cannot be upgraded: _Managed: Upgradeable: the apiserver is in a managed state, upgrades are not possible.
        * Cluster operator service-catalog-controller-manager cannot be upgraded: _Managed: Upgradeable: the controller manager is in a managed state, upgrades are not possible.
      reason: ClusterOperatorsNotUpgradeable
      status: "False"
      type: Upgradeable
    desired:
      force: false
      image: registry.svc.ci.openshift.org/ocp/release@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc
      version: ""
    history:
    - completionTime: null
      image: registry.svc.ci.openshift.org/ocp/release@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc
      startedTime: "2020-04-08T04:25:10Z"
      state: Partial
      verified: true
      version: ""
    - completionTime: "2020-04-08T04:17:10Z"
      image: registry.svc.ci.openshift.org/ocp/release@sha256:5e727bba8407a963fb2bdd95aaa2e2ba6aa63bc58da1f7e69ea28c3f43b90dea
      startedTime: "2020-04-08T04:16:55Z"
      state: Completed
      verified: false
      version: 4.4.0-0.nightly-2020-04-04-025830
    - completionTime: "2020-04-08T04:16:55Z"
      image: quay.io/openshift-release-dev/ocp-release-nightly@sha256:b7178a13e23d56e27647b3f2896a141af8b61ed83d8eca258ec95bc9bbeb92cc
      startedTime: "2020-04-08T04:08:40Z"
      state: Partial
      verified: true
      version: ""
    - completionTime: "2020-04-08T03:45:25Z"
      image: registry.svc.ci.openshift.org/ocp/release@sha256:5e727bba8407a963fb2bdd95aaa2e2ba6aa63bc58da1f7e69ea28c3f43b90dea
      startedTime: "2020-04-08T03:25:56Z"
      state: Completed
      verified: false
      version: 4.4.0-0.nightly-2020-04-04-025830
    observedGeneration: 4
    versionHash: ofdLDDmvxiQ=
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Comment 14 errata-xmlrpc 2020-05-04 11:29:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581