Bug 1499413

Paul,

I think this BZ is referring to Service Catalog functionality.

This has been partially addressed and the partial fix is present in https://github.com/openshift/origin/pull/16908

This sets a status.removedFromCatalog field is set when a service or plan is removed from the catalog.  A user will not be able to create new instances of a service or plan with this field set.

Fix is present in https://github.com/openshift/origin/pull/17027

This will be delivered into origin with https://github.com/openshift/origin/pull/17075

For the record, when a service class/plan is removed from the broker's payload, they won't necessarily be immediately removed from the catalog.  Instead, a status field saying that the service class/plan has been removed is set.  A control loop is always running looking for service classes/plans with this field set.  These control loops delete the resource when there are no instances left on a particular removed service class or plan.

So, if service class X is removed from the broker's catalog, and there are still two instances left on this service class, the service class resource won't be deleted until those two instances are deprovisioned.

Paul, I did below trying with latest images v3.7.0-0.189.0.0, but still cannot get the expect result.

[Scenario 1]: 
As the original description of this bug, I want to know how to deal with the invalid clusterserviceclass while clusterservicebroker change to a invalid spec.url. In my testing, those invalid clusterserviceclass were not deleted and removedFromBrokerCatalog always "false".

Steps:
1. Deploy svc-catalog & asb env by ansible-installer.
[root@qe-chezhang-master-etcd-1 ~]# oc get clusterserviceclass | grep -v NAME | wc -l
17
[root@qe-chezhang-master-etcd-1 ~]# oc describe clusterserviceclass | grep "Removed"
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false

2. Update clusterservicebroker spec.url to a invalid url
url: https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker
change to -->  url: https://invalid.asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker

3. Wait about 15mins, those invalid clusterserviceclass were not deleted and removedFromBrokerCatalog always "false"
[root@qe-chezhang-master-etcd-1 ~]# oc get clusterserviceclass | grep -v NAME | wc -l
17
[root@qe-chezhang-master-etcd-1 ~]# oc describe clusterserviceclass | grep "Removed"
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false


[Scenario 2]: 
Follow your comment 6, I used my personal registry and removed some apb image to see invalid clusterserviceclass and removedFromBrokerCatalog if will be change. 

Steps:
1. Check original clusterserviceclass state
[root@qe-chezhang-master-etcd-1 ~]# oc get clusterserviceclass | grep -v NAME | wc -l
5
[root@qe-chezhang-master-etcd-1 ~]# oc describe clusterserviceclass | grep "Removed"
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false

2. Provision only one serviceinstance with mediawiki apb

3. Remove mediawiki apb and postgresql apb images from docker-hub

3. Wait about 15mins, the two invalid clusterserviceclass were not deleted and removedFromBrokerCatalog always "false".
root@qe-chezhang-master-etcd-1 ~]# oc get clusterserviceclass | grep -v NAME | wc -l
5
[root@qe-chezhang-master-etcd-1 ~]# oc describe clusterserviceclass | grep "Removed"
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false
  Removed From Broker Catalog:	false

Paul, Please help to confirm my comment 8. Especially for [Scenario 1], user still can use those invalid clusterserviceclass from web console at that time.

In my understanding, that not looks good to me. Please assign to QA again and help to provide test scenario you expect if I have a mistake. Thanks.

Zhang Cheng-

Let me address the scenarios one at a time:

Scenario 1:

This scenario doesn't correctly make the service-catalog aware that any services have been removed from the broker catalog.  If you change the broker URL to an invalid one, no changes at all will be made to the service-catalog resources for ClusterServiceClass and ClusterServicePlan, because the broker cannot be contacted.  We wouldn't want this to be the case because we expect that intermittent broker outages are inevitable in time.  We would not want to remove and services or plans from the service-catalog just because we had trouble contacting the broker.  So, I'm not sure this is a valid scenario.

I'll address scenario 2 in my next comment; need to check with Erik Nelson on a couple things.

Commit pushed to master at https://github.com/openshift/origin

https://github.com/openshift/origin/commit/db1fdcb9115c296926bd28c62e90eaa30ba6f7dc
Merge pull request #17205 from eriknelson/catalog-status-roles

Automatic merge from submit-queue.

Add missing catalog service account roles

* Adds the ability for the catalog controller service account to modify
clusterserviceclass and clusterserviceplan status

Bug 1499413

If I'm understanding scenario #2 in comment 8 correctly, removing an image from docker hub is going to have no affect on anything. Secondly, provisioning an instance using a serviceclass/serviceplan from the ASB, you're guaranteeing that no deletions will occur as Paul explained earlier.

The correct way to ensure that serviceclasses and serviceplans are being properly removed is to:
- ensure you don't deploy any instances at all
- have the broker return different serviceclasses/serviceplans after they have been created in kubernetes (I'm not sure if the broker would necessarily support removing a service on the fly or if you'd have to modify the broker)

I'm not sure if this should really be a bug or if it should be changed to verify deletions do occur in the correct scenario as described above.

Test version is service-catalog & ansible-service-broker v3.9.0-0.20.0

Verified and passed. Detail info refer to comment 15.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3748