Red Hat Bugzilla – Bug 1534715
multiple DELETEs to the same binding result in multiple APB runs when async is turned on
Last modified: 2018-03-28 10:19:33 EDT
Copied from upstream: https://github.com/openshift/ansible-service-broker/issues/640 Description of problem: I initiated an unbind with the service catalog. The broker was configured for async bind and unbind. The catalog made multiple DELETE requests to the same endpoint: 172.17.0.2 - - [12/Jan/2018:21:17:47 +0000] "DELETE /ansible-service-broker/v2/service_instances/76593dc4-39d0-4561-a031-e0b4376e5532/service_bindings/7a477324-eb4c-4fb7-9aa6-64a52b2a688d?plan_id=7f4a5e35e4af2beb70076e72fab0b7ff&service_id=1dda1477cace09730bd8ed7a6505607e HTTP/1.1" 200 3 The second one returns an HTTP 500 response code: 172.17.0.2 - - [12/Jan/2018:21:17:57 +0000] "DELETE /ansible-service-broker/v2/service_instances/76593dc4-39d0-4561-a031-e0b4376e5532/service_bindings/7a477324-eb4c-4fb7-9aa6-64a52b2a688d?plan_id=7f4a5e35e4af2beb70076e72fab0b7ff&service_id=1dda1477cace09730bd8ed7a6505607e HTTP/1.1" 500 82 The broker logs make it clear that a second APB container was launched, and I was able to see both pods and both namespaces. Version-Release number of selected component (if applicable): current master branch of broker openshift v3.9.0-alpha.1+96f2f77-79 kubernetes v1.9.0-beta1 How reproducible: always Steps to Reproduce: 1. configure the broker for async bind and unbind 2. create a binding 3. delete the binding Actual results: multiple APBs run and some of the DELETE requests get a 500 response Expected results: only one APB runs, no 500 responses.
https://github.com/openshift/ansible-service-broker/pull/642
There is a bit more work to do on this it turns out. I'll have an additional PR shortly.
https://github.com/openshift/ansible-service-broker/pull/648
Michael, I encounter the same issue "launch_apb_on_bind is enabled, but accepts_incomplete is false, unbinding may fail" as I described comment3(step3) in the https://bugzilla.redhat.com/show_bug.cgi?id=1535955. So, the question is how to enable the "accepts_incomplete"? The setting below did not take effect. I'm sure I have restarted the controller-manager pod. [root@host-172-16-120-121 ~]# oc edit daemonset controller-manager -n kube-service-catalog containers: - args: - controller-manager - -v - "5" - --leader-election-namespace - kube-service-catalog - --broker-relist-interval - 5m - --feature-gates - OriginatingIdentity=true - --feature-gates - AsyncBindingOperations=true The warning info: [2018-01-26T07:24:47.347Z] [DEBUG] - Dao::DeleteBindInstance -> [ b8c86072-7b96-4b08-a3fb-a37abe0b471b ] 10.129.0.8 - - [26/Jan/2018:07:24:42 +0000] "DELETE /ansible-service-broker/v2/service_instances/550c4447-7a8e-4206-8ad2-3cd9f3ef8947/service_bindings/b8c86072-7b96-4b08-a3fb-a37abe0b471b?plan_id=9783fc2e859f9179833a7dd003baa841&service_id=d5915e05b253df421efe6e41fb6a66ba HTTP/1.1" 200 3 [2018-01-26T07:24:47.388Z] [WARNING] - launch_apb_on_bind is enabled, but accepts_incomplete is false, unbinding may fail [2018-01-26T07:24:47.393Z] [DEBUG] - Binding not found. 10.129.0.8 - - [26/Jan/2018:07:24:47 +0000] "DELETE /ansible-service-broker/v2/service_instances/550c4447-7a8e-4206-8ad2-3cd9f3ef8947/service_bindings/b8c86072-7b96-4b08-a3fb-a37abe0b471b?plan_id=9783fc2e859f9179833a7dd003baa841&service_id=d5915e05b253df421efe6e41fb6a66ba HTTP/1.1" 404 33 [2018-01-26T07:24:47.416Z] [WARNING] - launch_apb_on_bind is enabled, but accepts_incomplete is false, unbinding may fail [2018-01-26T07:24:47.423Z] [DEBUG] - Binding not found. The ASB/service-catalog image and version: [root@host-172-16-120-121 ~]# docker run --rm --entrypoint=asbd registry.access.stage.redhat.com/openshift3/ose-ansible-service-broker:v3.9.0-0.23.0.1 --version 1.1.7 [root@host-172-16-120-121 ~]# docker run --rm --entrypoint=service-catalog registry.reg-aws.openshift.com:443/openshift3/ose-service-catalog:v3.9 --version v0.1.3
Hi Jian, just to follow up, I've been using curl to directly make requests to the broker that include the query parameter "accepts_incomplete". I know you've seen those commands in little bash scripts this week on other issues, so just let me know if you have any other questions about it.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0489