Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1772392

Summary: ASB/TSB operators failed to upgrade from 4.2 to 4.3 for no "olm.skipRange" or "replace" field in CSVs
Product: OpenShift Container Platform Reporter: Cuiping HUO <chuo>
Component: Service BrokerAssignee: Fabian von Feilitzsch <fabian>
Status: CLOSED ERRATA QA Contact: Cuiping HUO <chuo>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.3.0CC: aos-bugs, chezhang, jesusr, jfan, jiazha
Target Milestone: ---Keywords: TestBlocker, Upgrades
Target Release: 4.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1772743 (view as bug list) Environment:
Last Closed: 2020-01-23 11:12:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1772743    
Attachments:
Description Flags
a detailed log of packagemanifest of redhat-operators-art and redhat-operators none

Description Cuiping HUO 2019-11-14 10:18:11 UTC
Created attachment 1636094 [details]
a detailed log of packagemanifest of redhat-operators-art and redhat-operators

Description of problem:
ASB/TSB operators failed to upgrade from 4.2 to 4.3 for no "olm.skipRange" or "replace" field in CSVs.

Version-Release number of selected component (if applicable):
4.3.0-0.nightly-2019-11-13-233341

How reproducible:
Always

Steps to Reproduce:
1. change app registry to redhat-operators-art
2. install tsb operators with manifest from redhat-operators-art, choose channel "4.2", the csv version is "openshifttemplateservicebrokeroperator.4.2.5-201911121709"
3. upgrade cluster from 4.2.5 to 4.3.0-0.nightly-2019-11-13-233341
4. change tsb operator subscription channel to "4.3"

Actual results:
the tsb operator subscription channel changed to "4.3", but the csv is still openshifttemplateservicebrokeroperator.4.2.5-201911121709 


Expected results:
the tsb operator subscription channel changed to "4.3", and the csv should be openshifttemplateservicebrokeroperator.openshiftansibleservicebroker.4.3.0-201911121117

Additional info:
checked csv of tsb, there is no "olm.skipRange" for channel "4.2"and "4.3",but "4.1" do have this field. Check with app registry:redhat-operators, there is also no "olm.skipRange" for channel "4.2" but "4.1" have. Attached a detailed log in the attachment.

$oc get packagemanifest openshifttemplateservicebroker -n openshift-marketplace -o yaml
apiVersion: packages.operators.coreos.com/v1
kind: PackageManifest
metadata:
  creationTimestamp: "2019-11-14T02:16:27Z"
  labels:
    catalog: qe-app-registry
    catalog-namespace: openshift-marketplace
    olm-visibility: hidden
    openshift-marketplace: "true"
    opsrc-datastore: "true"
    opsrc-owner-name: qe-app-registry
    opsrc-owner-namespace: openshift-marketplace
    provider: Red Hat, Inc.
    provider-url: ""
  name: openshifttemplateservicebroker
  namespace: openshift-marketplace
  selfLink: /apis/packages.operators.coreos.com/v1/namespaces/openshift-marketplace/packagemanifests/openshifttemplateservicebroker
spec: {}
status:
  catalogSource: qe-app-registry
  catalogSourceDisplayName: ""
  catalogSourceNamespace: openshift-marketplace
  catalogSourcePublisher: ""
  channels:
  - currentCSV: openshifttemplateservicebrokeroperator.4.2.5-201911121709
    currentCSVDesc:
      annotations:
        alm-examples: '[{"apiVersion":"osb.openshift.io/v1","kind":"TemplateServiceBroker","metadata":{"name":"template-service-broker","namespace":"template-service-broker"},"spec":{}}]'
        capabilities: Seamless Upgrades
        containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker@sha256:4b33168a9d5963c83f678f1efc0291560ff2689ee3b3f4a1b72d10d02f89a666
        description: OpenShift Template Service Broker is an implementation of the
          [Open Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
      apiservicedefinitions: {}
      customresourcedefinitions:
        owned:
        - description: An Open Service Broker supporting management of OpenShift templates.
          displayName: Template Service Broker
          kind: TemplateServiceBroker
          name: templateservicebrokers.osb.openshift.io
          version: v1
      description: |
        The OpenShift Template Service Broker implements the [Open Service Broker
        API](https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md)
        endpoints:
        - *Catalog*: returns a list of available templates as OSB API
          *Service* objects (the templates are read from one or more
          namespaces configured in the master config).
        - *Provision*: provision a given template (referred by its UID) into a
          namespace.  Under the covers, this creates a non-namespaced
          **BrokerTemplateInstance** object for the template service broker to
          store state associated with the the instantiation, as well as the
          **Secret** and **TemplateInstance** objects which are picked up by
          the **TemplateInstance** controller.  *Provision* is an asynchronous
          operation: it may return before provisioning is completed, and the
          provision status can (must) be recovered via the *Last Operation*
          endpoint (see below).
        - *Bind*: for a given template, return "credentials" exposed in any
          created ConfigMap, Secret, Service or Route object (see
          ExposeAnnotationPrefix and Base64ExposeAnnotationPrefix
          documentation).  The *Bind* call records the fact that it took
          place in the appropriate **BrokerTemplateInstance** object.
        - *Unbind*: this simply removes the metadata previously placed in the
          **BrokerTemplateInstance** object by a *Bind* call.
        - *Deprovision*: removes the objects created by the *Provision* call.
          The garbage collector removes all additional objects created by the
          **TemplateInstance** controller, hopefully transitively, as
          documented above.
        - *Last Operation*: returns the status of the previously run
          asynchronous operation.  In the template service broker, *Provision*
          is the only asynchronous operation.
      displayName: OpenShift Template Service Broker Operator
      installModes:
      - supported: true
        type: OwnNamespace
      - supported: true
        type: SingleNamespace
      - supported: false
        type: MultiNamespace
      - supported: false
        type: AllNamespaces
      provider:
        name: Red Hat, Inc.
      version: 4.2.5-201911121709
    name: "4.2"
  - currentCSV: openshifttemplateservicebrokeroperator.4.3.0-201911081022
    currentCSVDesc:
      annotations:
        alm-examples: '[{"apiVersion":"osb.openshift.io/v1","kind":"TemplateServiceBroker","metadata":{"name":"template-service-broker","namespace":"template-service-broker"},"spec":{}}]'
        capabilities: Seamless Upgrades
        containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker@sha256:e79b20346de0faa6e807725ed2989de9aa3d2401a97b77c256bb8359ece316d7
        description: OpenShift Template Service Broker is an implementation of the
          [Open Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
      apiservicedefinitions: {}
      customresourcedefinitions:
        owned:
        - description: An Open Service Broker supporting management of OpenShift templates.
          displayName: Template Service Broker
          kind: TemplateServiceBroker
          name: templateservicebrokers.osb.openshift.io
          version: v1
      description: |
        The OpenShift Template Service Broker implements the [Open Service Broker
        API](https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md)
        endpoints:
        - *Catalog*: returns a list of available templates as OSB API
          *Service* objects (the templates are read from one or more
          namespaces configured in the master config).
        - *Provision*: provision a given template (referred by its UID) into a
          namespace.  Under the covers, this creates a non-namespaced
          **BrokerTemplateInstance** object for the template service broker to
          store state associated with the the instantiation, as well as the
          **Secret** and **TemplateInstance** objects which are picked up by
          the **TemplateInstance** controller.  *Provision* is an asynchronous
          operation: it may return before provisioning is completed, and the
          provision status can (must) be recovered via the *Last Operation*
          endpoint (see below).
        - *Bind*: for a given template, return "credentials" exposed in any
          created ConfigMap, Secret, Service or Route object (see
          ExposeAnnotationPrefix and Base64ExposeAnnotationPrefix
          documentation).  The *Bind* call records the fact that it took
          place in the appropriate **BrokerTemplateInstance** object.
        - *Unbind*: this simply removes the metadata previously placed in the
          **BrokerTemplateInstance** object by a *Bind* call.
        - *Deprovision*: removes the objects created by the *Provision* call.
          The garbage collector removes all additional objects created by the
          **TemplateInstance** controller, hopefully transitively, as
          documented above.
        - *Last Operation*: returns the status of the previously run
          asynchronous operation.  In the template service broker, *Provision*
          is the only asynchronous operation.
      displayName: OpenShift Template Service Broker Operator
      installModes:
      - supported: true
        type: OwnNamespace
      - supported: true
        type: SingleNamespace
      - supported: false
        type: MultiNamespace
      - supported: false
        type: AllNamespaces
      provider:
        name: Red Hat, Inc.
      version: 4.3.0-201911081022
    name: "4.3"
  - currentCSV: openshifttemplateservicebrokeroperator.4.1.24-201911120311
    currentCSVDesc:
      annotations:
        alm-examples: '[{"apiVersion":"osb.openshift.io/v1","kind":"TemplateServiceBroker","metadata":{"name":"template-service-broker","namespace":"template-service-broker"},"spec":{}}]'
        capabilities: Seamless Upgrades
        containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker@sha256:b76eaf9a9a73ed95f3c596e1f3723c9ebd4fa9648a4958099c39ac2008058673
        description: OpenShift Template Service Broker is an implementation of the
          [Open Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
        olm.skipRange: '>=4.1.0 <4.1.24-201911120311'
      apiservicedefinitions: {}

Comment 1 Cuiping HUO 2019-11-15 02:28:37 UTC
@fvonfeil,
TAB has same issue, we also need tsb to be fixed.

Comment 2 Cuiping HUO 2019-11-15 02:29:13 UTC
(In reply to Cuiping HUO from comment #1)
> @fvonfeil,
> TAB has same issue, we also need tsb to be fixed.

should be TSB.

Comment 4 Cuiping HUO 2019-11-18 10:12:54 UTC
app registry redhat-operators-art did not ready for 4.2.
For asb:
The 4.3 csv is already have field olm.skipRange: '>=4.2.0 <4.3.0-201911151517' but 4.2 csv still not contain field'olm.skipRange'

$ oc get csv -n openshift-ansible-service-broker openshiftansibleservicebroker.4.3.0-201911151517 -o yaml
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  annotations:
    alm-examples: '[{"apiVersion":"osb.openshift.io/v1", "kind":"AutomationBroker",
      "metadata":{"name":"ansible-service-broker","namespace":"ansible-service-broker"},
      "spec":{"createBrokerNamespace":"false","waitForBroker":"false", "registries":
      [{"type": "rhcc", "name": "rhcc", "url": "https://registry.redhat.io", "white_list":
      [".*-apb$"], "auth_type": "secret", "auth_name": "asb-registry-auth"}]}}]'
    capabilities: Seamless Upgrades
    containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker@sha256:b63bd391d71ecb7c49da6d70d52f1d8b2758408f10c243fcb454db09d9bf3534
    description: OpenShift Ansible Service Broker is an implementation of the [Open
      Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
    olm.operatorGroup: openshift-ansible-service-broker-zpgfc
    olm.operatorNamespace: openshift-ansible-service-broker
    olm.skipRange: '>=4.2.0 <4.3.0-201911151517'
$ oc get csv -n openshift-ansible-service-broker openshiftansibleservicebroker.4.2.7-201911150432 -o yaml
apiVersion: v1
items:
- apiVersion: operators.coreos.com/v1alpha1
  kind: ClusterServiceVersion
  metadata:
    annotations:
      alm-examples: '[{"apiVersion":"osb.openshift.io/v1", "kind":"AutomationBroker",
        "metadata":{"name":"ansible-service-broker","namespace":"ansible-service-broker"},
        "spec":{"createBrokerNamespace":"false","waitForBroker":"false", "registries":
        [{"type": "rhcc", "name": "rhcc", "url": "https://registry.redhat.io", "white_list":
        [".*-apb$"], "auth_type": "secret", "auth_name": "asb-registry-auth"}]}}]'
      capabilities: Seamless Upgrades
      containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker@sha256:cef7150704639b933a90c7bacb1f11dd49a3b325c9f4cd1e227227d1c03b1034
      description: OpenShift Ansible Service Broker is an implementation of the [Open
        Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
      olm.operatorGroup: openshift-ansible-service-broker-zpgfc
      olm.operatorNamespace: openshift-ansible-service-broker
      olm.targetNamespaces: openshift-ansible-service-broker
    creationTimestamp: "2019-11-18T09:58:54Z"
    generation: 1
    labels:
      olm.api.2fc6c4e2d95a77f0: provided
      olm.api.880c523f5d0ea900: provided
      olm.api.7044497f8b7e10a9: provided
      olm.api.dba600531489ce6c: provided
    name: openshiftansibleservicebroker.4.2.7-201911150432
    namespace: openshift-ansible-service-broker
    resourceVersion: "83132"
    selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/openshift-ansible-service-broker/clusterserviceversions/openshiftansibleservicebroker.4.2.7-201911150432
    uid: 8c28ab96-5ed6-4399-8307-338dda66b049

$ oc get opsrc -n openshift-marketplace
NAME                  TYPE          ENDPOINT              REGISTRY               DISPLAYNAME           PUBLISHER   STATUS      MESSAGE                                       AGE
qe-app-registry       appregistry   https://quay.io/cnr   redhat-operators-art   RedHat Operators      Red Hat     Succeeded   The object has been successfully reconciled   83m

Comment 5 Jesus M. Rodriguez 2019-11-18 15:09:47 UTC
This bug needs to get verified BEFORE the 4.2 bug can be fixed: https://bugzilla.redhat.com/show_bug.cgi?id=1772743

If the 4.3 has the correct skipRange, then I would say this bug is done. Then we can cherry-pick the fix to the 4.2 branch.

Comment 6 Cuiping HUO 2019-11-19 02:44:56 UTC
Verified.
cluster version: 4.3.0-0.nightly-2019-11-18-175710	
The 4.3 csv of asb and tsb have field olm.skipRange correctly.

$ oc get csv -n openshift-ansible-service-broker openshiftansibleservicebroker.4.3.0-201911151517 -o yaml
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  annotations:
    alm-examples: '[{"apiVersion":"osb.openshift.io/v1", "kind":"AutomationBroker",
      "metadata":{"name":"ansible-service-broker","namespace":"ansible-service-broker"},
      "spec":{"createBrokerNamespace":"false","waitForBroker":"false", "registries":
      [{"type": "rhcc", "name": "rhcc", "url": "https://registry.redhat.io", "white_list":
      [".*-apb$"], "auth_type": "secret", "auth_name": "asb-registry-auth"}]}}]'
    capabilities: Seamless Upgrades
    containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker@sha256:b63bd391d71ecb7c49da6d70d52f1d8b2758408f10c243fcb454db09d9bf3534
    description: OpenShift Ansible Service Broker is an implementation of the [Open
      Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
    olm.operatorGroup: openshift-ansible-service-broker-zpgfc
    olm.operatorNamespace: openshift-ansible-service-broker
    olm.skipRange: '>=4.2.0 <4.3.0-201911151517'


$ oc get csv openshifttemplateservicebrokeroperator.4.3.0-201911161914 -n openshift-template-service-broker -o yaml
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  annotations:
    alm-examples: '[{"apiVersion":"osb.openshift.io/v1","kind":"TemplateServiceBroker","metadata":{"name":"template-service-broker","namespace":"template-service-broker"},"spec":{}}]'
    capabilities: Seamless Upgrades
    containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker@sha256:e3faf33653482f7f73b4e540d174d57bfccb17ce9bea54e0a59d89a4e357c4be
    description: OpenShift Template Service Broker is an implementation of the [Open
      Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
    olm.operatorGroup: openshift-template-service-broker-np69w
    olm.operatorNamespace: openshift-template-service-broker
    olm.skipRange: '>=4.2.0 <4.3.0-201911161914'
    olm.targetNamespaces: openshift-template-service-broker
  creationTimestamp: "2019-11-19T02:31:32Z"
  generation: 1
  labels:
    olm.api.57b56b9c59508420: provided
  name: openshifttemplateservicebrokeroperator.4.3.0-201911161914
  namespace: openshift-template-service-broker

Comment 8 errata-xmlrpc 2020-01-23 11:12:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062