Bug 1500615 - should see serviceclass relist by RelistDuration setting while setting RelistDuration greater than 5mins
Summary: should see serviceclass relist by RelistDuration setting while setting Relist...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Service Broker
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.7.0
Assignee: Jay Boyd
QA Contact: Zhang Cheng
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-11 08:16 UTC by Zhang Cheng
Modified: 2017-11-28 22:15 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2017-11-28 22:15:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:3188 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-29 02:34:54 UTC

Comment 1 Erik Nelson 2017-10-17 18:46:07 UTC
After some investigation into this, I believe the observed behavior is actually expected given the circumstances.

A change was recently merged to the Catalog that exposed a bug in the ASB's /catalog response (the response was non-compliant with the OSB spec). This bad response caused the Catalog to error during ServiceClass reconciliation. It is expected that the catalog will continue to try to relist the broker *despite a Manual RelistBehavior*, if it has not successfully reconciled.

After setting the broker-config configmap's "refresh_interval" to "24h", and placing the clusterservicebroker into "Manual" mode with no duration, I have not seen any request activity in the ASB logs for hours.

Closing as NOTABUG, if anyone feels differently, please reopen for further investigation.

Comment 3 Erik Nelson 2017-11-01 18:59:36 UTC
Reopened this as a bug because I was able to confirm the relist requests are hitting brokers on 5 minute intervals despite having RelistDuration settings of greater than 5 minutes.

The root cause is that lastTransitionTime is not updated for every successful service reconciliation, so the interval appears to have elapsed since the *initial* setting of lastTransitionTime, causing the catalog to relist on the default resync duration of 5m.

The fix for this is to make sure lastTransitionTime is updated after every successful relist.

NOTE: "resync" messages, as seen in the logs, are not themselves and indicator that a "relist" has occurred.

Comment 4 Zhang Cheng 2017-11-02 08:43:38 UTC
I updated title of this bug base on Erik's comments. 

Also tried with latest image again, and confirmed with:
1. No any serviceclass relists(/catalog requests) on an interval with relistBehavior set to Manual

2. When setting RelistDuration greater than 5mins, can get /catalog requests, but always using 5mins, I hope to use this bug to trace this problem.

Config in broker:
  relistBehavior: Duration
  relistDuration: 15m0s

Log of asb pod:
[2017-11-02T08:27:31.306Z] [INFO] AnsibleBroker::Catalog
10.129.0.1 - - [02/Nov/2017:08:27:31 +0000] "GET /ansible-service-broker/v2/catalog HTTP/1.1" 200 74264
[2017-11-02T08:32:31.305Z] [INFO] AnsibleBroker::Catalog
10.129.0.1 - - [02/Nov/2017:08:32:31 +0000] "GET /ansible-service-broker/v2/catalog HTTP/1.1" 200 74264

Comment 5 Jay Boyd 2017-11-02 13:54:54 UTC
I've reproduced the issue and see the problem.  Thanks Erik, good info in comment #3.  We're addressing upstream and should have this fixed today.

Comment 6 Jay Boyd 2017-11-03 15:47:57 UTC
fixed upstream by https://github.com/kubernetes-incubator/service-catalog/issues/1513 and merged into OpenShift by https://github.com/openshift/origin/pull/17166

Comment 9 Zhang Cheng 2017-11-04 16:16:37 UTC
Test images should be v3.7.0-0.191.0.0

Comment 12 errata-xmlrpc 2017-11-28 22:15:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188


Note You need to log in before you can comment on or make changes to this bug.