Created attachment 1383609 [details] script to create a binding Reported upstream: https://github.com/openshift/ansible-service-broker/issues/667 Description of problem: I used the broker API to create a binding by making this request: PUT /ansible-service-broker/v2/service_instances/c4aa708d-8c64-4c37-a131-557a4a155148/service_bindings/745ddeb7-395e-412e-b4ed-8a0b52b11ac0?accepts_incomplete=true The response code was 201, but should have been 202 according to the OSB API spec proposed changes around async bind/unbind. HTTP/1.1 201 Created Content-Type: application/json Date: Fri, 19 Jan 2018 19:42:53 GMT Content-Length: 58 Set-Cookie: d05351dd24520dce41f310fcfc112523=1591f023a03f3453b1d2364f14c83799; path=/; HttpOnly; Secure { "operation": "338c289e-8792-4ab4-95fa-4fd9b4752cf0" } Version-Release number of selected component (if applicable): master How reproducible: always Steps to Reproduce: 1. configure the broker to run APBs on bind/unbind and auto-escalate 2. Run the attached script, supplying a valid instance ID as the only argument Actual results: 201 response code Expected results: 202 response code
https://github.com/openshift/ansible-service-broker/pull/669
Apologies. I left out the detail that you need to use an APB that supports async bind and unbind. Currently there's just an experimental one in my repo: https://hub.docker.com/r/mhrivnak/postgresql-apb/
Michael, I used the example APB you're provided, configure the below: - type: dockerhub name: dh url: https://registry.hub.docker.com org: mhrivnak tag: latest white_list: - ".*-apb$" The ASB version: 1.1.9 Test steps as the above, the response of async binding is 202, looks good to me. [root@localhost jzhang]# ./bind.sh binding ID: 40a35cdd-5f51-4559-ab79-6a685709cd96 HTTP/1.1 202 Accepted Content-Type: application/json Date: Fri, 09 Feb 2018 02:58:37 GMT Content-Length: 58 Set-Cookie: 89a6d633054ded194d4e1360cdc1fbef=b37e683520b874bc37db0badfda2d6dd; path=/; HttpOnly; Secure { "operation": "b4f065ed-2010-47b0-8d61-3bc438f63ac3" } Check the state: [root@localhost jzhang]# ./last_operation.sh b4f065ed-2010-47b0-8d61-3bc438f63ac3 { "state": "succeeded" } [root@localhost jzhang]# cat last_operation.sh OPERATION_ID=$1 INSTANCE_ID="d10a0ca6-c609-4d29-ab9d-b25c3b8c332c" PLAN_UUID="7f4a5e35e4af2beb70076e72fab0b7ff" SERVICE_UUID="dh-postgresql-apb-b7gbr" IDENTITY=$(echo -n '{"groups":["system:authenticated:oauth","system:authenticated"],"scopes.authorization.openshift.io":["user:full"],"uid":"","username":"jiazha"}'|base64 -w 0) curl \ -k \ -X GET \ -H "Authorization: bearer $(oc whoami -t)" \ -H "Content-type: application/json" \ -H "Accept: application/json" \ -H "X-Broker-API-Originating-Identity: kubernetes $IDENTITY" \ "https://asb-1338-openshift-ansible-service-broker.apps.0207-nvc.qe.rhcloud.com/ansible-service-broker/v2/service_instances/$INSTANCE_ID/last_operation?operation=$OPERATION_ID&service_id=$SERVICE_UUID&plan_id=$PLAN_UUID" But, I didn't find the created binding although it returned the "succeed" status. Why? I think the users should get the servicebindings. [root@host-172-16-120-54 ~]# oc get servicebinding --all-namespaces No resources found. And, according to comment 9, I want to know whether our released APBs(such as PostgreSQL, MariaDB, MySQL, Mediawiki) will support the ASYNC in 3.9?
Michael, Another thing is your example APB failed when running async unbind. Details as below: For creating binding steps, please refer to the above. Run the unbind action by using below script: [root@localhost jzhang]# cat unbind.sh INSTANCE_ID="0a0b861b-3045-4d98-b137-71d1db572d2e" BINDING_ID=$1 PLAN_UUID="7f4a5e35e4af2beb70076e72fab0b7ff" IDENTITY=$(echo -n '{"groups":["system:authenticated:oauth","system:authenticated"],"scopes.authorization.openshift.io":["user:full"],"uid":"","username":"jiazha"}'|base64 -w 0) curl \ -k -i \ -X DELETE \ -H "Authorization: bearer $(oc whoami -t)" \ -H "Content-type: application/json" \ -H "Accept: application/json" \ -H "X-Broker-API-Originating-Identity: kubernetes $IDENTITY" \ "https://asb-1338-openshift-ansible-service-broker.apps.0207-nvc.qe.rhcloud.com/ansible-service-broker/v2/service_instances/$INSTANCE_ID/service_bindings/$BINDING_ID?plan_id=$PLAN_UUID" [root@localhost jzhang]# ./unbind.sh ef42a7b3-1d74-4e43-9447-a0af58719c55 HTTP/1.1 500 Internal Server Error Content-Type: application/json Date: Fri, 09 Feb 2018 06:46:03 GMT Content-Length: 98 Set-Cookie: 89a6d633054ded194d4e1360cdc1fbef=45b7fdf950d87e8c3a786f6dc8075bc5; path=/; HttpOnly; Secure { "description": "Pod [ apb-2a8a1fdb-5324-4459-846a-ff3fcf218787 ] failed with exit code [2]" } The ASB related logs: [2018-02-09T06:46:03.652Z] [ERROR] - Unbind action failed - Pod [ apb-2a8a1fdb-5324-4459-846a-ff3fcf218787 ] failed with exit code [2] [2018-02-09T06:46:03.652Z] [INFO] - Destroying APB sandbox... [2018-02-09T06:46:03.655Z] [DEBUG] - Keeping namespace alive due to configuration [2018-02-09T06:46:03.655Z] [DEBUG] - Deleting rolebinding apb-2a8a1fdb-5324-4459-846a-ff3fcf218787, namespace dh-postgresql-apb-unbi-qsrnn [2018-02-09T06:46:03.658Z] [NOTICE] - Successfully deleted rolebinding apb-2a8a1fdb-5324-4459-846a-ff3fcf218787, namespace dh-postgresql-apb-unbi-qsrnn [2018-02-09T06:46:03.658Z] [DEBUG] - Deleting rolebinding apb-2a8a1fdb-5324-4459-846a-ff3fcf218787, namespace async3 [2018-02-09T06:46:03.664Z] [NOTICE] - Successfully deleted rolebinding apb-2a8a1fdb-5324-4459-846a-ff3fcf218787, namespace async3 [2018-02-09T06:46:03.664Z] [DEBUG] - Deleting network policy for pod: apb-2a8a1fdb-5324-4459-846a-ff3fcf218787 to grant network access to ns: async3 [2018-02-09T06:46:03.671Z] [DEBUG] - Successfully deleted network policy for pod: apb-2a8a1fdb-5324-4459-846a-ff3fcf218787 to grant network access to ns: async3 [2018-02-09T06:46:03.671Z] [DEBUG] - Running post sandbox destroy hooks [2018-02-09T06:46:03.671Z] [ERROR] - Unknown error: Pod [ apb-2a8a1fdb-5324-4459-846a-ff3fcf218787 ] failed with exit code [2] 10.128.0.1 - - [09/Feb/2018:06:45:53 +0000] "DELETE /ansible-service-broker/v2/service_instances/0a0b861b-3045-4d98-b137-71d1db572d2e/service_bindings/ef42a7b3-1d74-4e43-9447-a0af58719c55?plan_id=7f4a5e35e4af2beb70076e72fab0b7ff HTTP/1.1" 500 98 I intended to new a bug for tracking this, but as you know, the above example APB is NOT our released version, so, how do you think?
(In reply to Jian Zhang from comment #10) > > But, I didn't find the created binding although it returned the "succeed" > status. Why? I think the users should get the servicebindings. > [root@host-172-16-120-54 ~]# oc get servicebinding --all-namespaces > No resources found. That's expected. According to the OSB API spec, the "Platform", which in our case is the console, is responsible for managing its records of service instances and bindings. While our broker does happen to manage resources within the cluster for the sake of provisioning services, that should not be confused with managing state on behalf of the "Platform". It took me a while for that to sink in. This Terminology section may help clear things up: https://github.com/openservicebrokerapi/servicebroker/blob/v2.13/spec.md#terminology > > And, according to comment 9, I want to know whether our released APBs(such > as PostgreSQL, MariaDB, MySQL, Mediawiki) will support the ASYNC in 3.9? My understanding is that we are not planning to have async bind and unbind officially supported by an APB in 3.9. But, we are looking for a way to make that feature available for testing and experimentation, perhaps by having an unofficial or upstream APB available elsewhere. Having an image in my personal docker hub org is clearly not the best, so suggestions are welcome!
(In reply to Jian Zhang from comment #11) > Michael, > > Another thing is your example APB failed when running async unbind. Ok, feel free to NEEDSINFO me when you file that bug. I've found the best (and maybe only) way to get more detail about such a failure is to enable the broker's feature "keep_namespace_on_error: True", and then go look at the logs for the APB pod. It should show you the output from ansible.
Michael, Thanks for your clear explanation! I got it. As for the storing of the example APB, I think maybe we can put it on the https://hub.docker.com/r/ansibleplaybookbundle. And, we can rename or label it with "async experimentation" or something. For the unbind failure, I could NOT reproduce it with the latest version. The ASB version: 1.1.10 reproduce steps: 1, Enable the async of the ASB and enable the develop mode, like below: openshift: ... keep_namespace_on_error: true broker: ... dev_broker: true launch_apb_on_bind: true 2, Config the registry which stored the example async APB, like: registry: - type: dockerhub name: dh url: https://registry.hub.docker.com org: mhrivnak tag: latest white_list: - ".*-apb$" 3, New a porject and provision the example APB(PostgreSQL) in webconsole. [root@host-172-16-120-48 ~]# oc get pods -n jian NAME READY STATUS RESTARTS AGE postgresql-9.6-dev-1-vtxm6 1/1 Running 0 9m [root@host-172-16-120-48 ~]# oc get serviceinstance -n jian NAME AGE dh-postgresql-apb-5mmtr 10m 4, Create a binding by using the script below and replace the related values with your own. [jzhang@localhost ~]$ ./bind.sh binding ID: 0683134d-5f4d-41ae-b32b-bb7108d3021e HTTP/1.1 202 Accepted Content-Type: application/json Date: Sun, 11 Feb 2018 03:53:43 GMT Content-Length: 58 Set-Cookie: 89a6d633054ded194d4e1360cdc1fbef=a3d8c4c08e2850819c78cf452f68a11f; path=/; HttpOnly; Secure { "operation": "96232702-20ab-469e-830f-4e1a1fb212a1" } 5, Check status: [jzhang@localhost ~]$ ./last_operation.sh 96232702-20ab-469e-830f-4e1a1fb212a1 { "state": "succeeded" } 6, Run the async unbind action, looks worked well! [jzhang@localhost ~]$ ./unbind.sh 0683134d-5f4d-41ae-b32b-bb7108d3021e HTTP/1.1 202 Accepted Content-Type: application/json Date: Sun, 11 Feb 2018 03:54:11 GMT Content-Length: 58 Set-Cookie: 89a6d633054ded194d4e1360cdc1fbef=a3d8c4c08e2850819c78cf452f68a11f; path=/; HttpOnly; Secure { "operation": "11c2110b-45a3-472c-9a80-fbf1cad0c22f" } [jzhang@localhost ~]$ cat unbind.sh INSTANCE_ID="3ac081f5-0605-4462-b354-127a99466ce1" BINDING_ID=$1 PLAN_UUID="7f4a5e35e4af2beb70076e72fab0b7ff" IDENTITY=$(echo -n '{"groups":["system:authenticated:oauth","system:authenticated"],"scopes.authorization.openshift.io":["user:full"],"uid":"","username":"jiazha"}'|base64 -w 0) curl \ -k -i \ -X DELETE \ -H "Authorization: bearer $(oc whoami -t)" \ -H "Content-type: application/json" \ -H "Accept: application/json" \ -H "X-Broker-API-Originating-Identity: kubernetes $IDENTITY" \ "https://asb-1338-openshift-ansible-service-broker.apps.0211-dxf.qe.rhcloud.com/ansible-service-broker/v2/service_instances/$INSTANCE_ID/service_bindings/$BINDING_ID?accepts_incomplete=true&plan_id=$PLAN_UUID" The ASB logs: ... [2018-02-11T03:54:11.787Z] [INFO] - ASYNC unbinding in progress [2018-02-11T03:54:11.787Z] [DEBUG] - unbindjob: unbinding job (11c2110b-45a3-472c-9a80-fbf1cad0c22f) started, calling apb.Unbind [2018-02-11T03:54:11.787Z] [NOTICE] - ============================================================ [2018-02-11T03:54:11.787Z] [NOTICE] - UNBINDING [2018-02-11T03:54:11.787Z] [NOTICE] - ============================================================ [2018-02-11T03:54:11.787Z] [NOTICE] - ServiceInstance.ID: 1dda1477cace09730bd8ed7a6505607e [2018-02-11T03:54:11.787Z] [NOTICE] - ServiceInstance.Name: dh-postgresql-apb [2018-02-11T03:54:11.787Z] [NOTICE] - ServiceInstance.Image: docker.io/mhrivnak/postgresql-apb:latest [2018-02-11T03:54:11.787Z] [NOTICE] - ServiceInstance.Description: SCL PostgreSQL apb implementation [2018-02-11T03:54:11.787Z] [NOTICE] - ============================================================ [2018-02-11T03:54:11.787Z] [DEBUG] - ExecutingApb: [2018-02-11T03:54:11.787Z] [DEBUG] - name:[ dh-postgresql-apb ] [2018-02-11T03:54:11.787Z] [DEBUG] - image:[ docker.io/mhrivnak/postgresql-apb:latest ] [2018-02-11T03:54:11.787Z] [DEBUG] - action:[ unbind ] [2018-02-11T03:54:11.787Z] [DEBUG] - pullPolicy:[ IfNotPresent ] [2018-02-11T03:54:11.787Z] [DEBUG] - role:[ edit ] [2018-02-11T03:54:11.787Z] [DEBUG] - No proxy env vars found to be configured. [2018-02-11T03:54:11.811Z] [DEBUG] - Dao::DeleteBindInstance -> [ 0683134d-5f4d-41ae-b32b-bb7108d3021e ] 10.129.0.1 - - [11/Feb/2018:03:54:11 +0000] "DELETE /ansible-service-broker/v2/service_instances/3ac081f5-0605-4462-b354-127a99466ce1/service_bindings/0683134d-5f4d-41ae-b32b-bb7108d3021e?accepts_incomplete=true&plan_id=7f4a5e35e4af2beb70076e72fab0b7ff HTTP/1.1" 202 58 [2018-02-11T03:54:11.887Z] [DEBUG] - Trying to create apb sandbox: [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ], with edit permissions in namespace dh-postgresql-apb-unbi-swj7s [2018-02-11T03:54:11.887Z] [NOTICE] - Creating RoleBinding apb-333ca488-fb13-4115-b498-6f416dd79d72 [2018-02-11T03:54:12.033Z] [NOTICE] - Creating RoleBinding apb-333ca488-fb13-4115-b498-6f416dd79d72 [2018-02-11T03:54:12.091Z] [DEBUG] - Creating network policy for pod: apb-333ca488-fb13-4115-b498-6f416dd79d72 to grant network access to ns: jian [2018-02-11T03:54:12.096Z] [DEBUG] - Successfully created network policy for pod: apb-333ca488-fb13-4115-b498-6f416dd79d72 to grant network access to ns: jian [2018-02-11T03:54:12.096Z] [INFO] - Successfully created apb sandbox: [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ], with edit permissions in namespace dh-postgresql-apb-unbi-swj7s [2018-02-11T03:54:12.096Z] [INFO] - Running post create sandbox fuctions if defined. [2018-02-11T03:54:12.096Z] [NOTICE] - Creating pod "apb-333ca488-fb13-4115-b498-6f416dd79d72" in the dh-postgresql-apb-unbi-swj7s namespace [2018-02-11T03:54:12.11Z] [DEBUG] - Watching pod [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ] in namespace [ dh-postgresql-apb-unbi-swj7s ] for completion [2018-02-11T03:54:12.11Z] [INFO] - Watch pod [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ] tick 1 [2018-02-11T03:54:12.127Z] [DEBUG] - Pod [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ] Pending [2018-02-11T03:54:17.13Z] [INFO] - Watch pod [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ] tick 2 [2018-02-11T03:54:17.156Z] [DEBUG] - Pod [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ] Running [2018-02-11T03:54:22.156Z] [INFO] - Watch pod [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ] tick 3 [2018-02-11T03:54:22.16Z] [DEBUG] - Pod [ apb-333ca488-fb13-4115-b498-6f416dd79d72 ] completed [2018-02-11T03:54:22.16Z] [INFO] - Destroying APB sandbox... [2018-02-11T03:54:22.162Z] [DEBUG] - Deleting namespace dh-postgresql-apb-unbi-swj7s [2018-02-11T03:54:22.168Z] [DEBUG] - Deleting rolebinding apb-333ca488-fb13-4115-b498-6f416dd79d72, namespace dh-postgresql-apb-unbi-swj7s [2018-02-11T03:54:22.182Z] [NOTICE] - Successfully deleted rolebinding apb-333ca488-fb13-4115-b498-6f416dd79d72, namespace dh-postgresql-apb-unbi-swj7s [2018-02-11T03:54:22.182Z] [DEBUG] - Deleting rolebinding apb-333ca488-fb13-4115-b498-6f416dd79d72, namespace jian [2018-02-11T03:54:22.186Z] [NOTICE] - Successfully deleted rolebinding apb-333ca488-fb13-4115-b498-6f416dd79d72, namespace jian [2018-02-11T03:54:22.186Z] [DEBUG] - Deleting network policy for pod: apb-333ca488-fb13-4115-b498-6f416dd79d72 to grant network access to ns: jian [2018-02-11T03:54:22.193Z] [DEBUG] - Successfully deleted network policy for pod: apb-333ca488-fb13-4115-b498-6f416dd79d72 to grant network access to ns: jian [2018-02-11T03:54:22.193Z] [DEBUG] - Running post sandbox destroy hooks [2018-02-11T03:54:22.193Z] [DEBUG] - unbindjob: returned from apb.Unbind [2018-02-11T03:54:22.193Z] [DEBUG] - unbindjob: Looks like we're done [2018-02-11T03:54:22.193Z] [DEBUG] - Processed binding message from buffer At present, the async unbind works well, I will keep an eye on it. And, for this fix, it looks good to me, thanks!
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0489