Bug 1676760

Summary: Failed to install ASB via marketplace
Product: OpenShift Container Platform Reporter: Zihan Tang <zitang>
Component: Service BrokerAssignee: Alay Patel <alpatel>
Status: CLOSED ERRATA QA Contact: Zihan Tang <zitang>
Severity: high Docs Contact:
Priority: high    
Version: 4.1.0CC: aos-bugs, chezhang, shurley, sponnaga
Target Milestone: ---Keywords: Reopened
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:44:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Zihan Tang 2019-02-13 06:36:10 UTC
Description of problem:
When using operators from operator hub(marketplace) to install asb, it failed at create CR with errors:
        An unhandled exception occurred while running the lookup plugin 'k8s'. Error was a <class 'kubernetes.client.rest.ApiException'>, original message: (401)
        Reason: Unauthorized
        HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 13 Feb 2019 03:45:12 GMT', 'Content-Length': '129', 'Content-Type': 'application/json', 'Cache-Control': 'no-store'})
        HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}


How reproducible:
always

Steps to Reproduce:
1. install service catalog from operator hub to 'openshift-operators' project
operator hub->show community operaotrs->chooose 'service catalog'-> install->choose 'global operaots' og in Target -> click 'subscribe'

 [core@ip-10-0-8-250 ~]$ oc get pod -n openshift-operators
NAME                                       READY   STATUS    RESTARTS   AGE
svcat-apiserver-554585dc45-tcwvf           2/2     Running   0          53m
svcat-controller-manager-b85874ff6-sc2ns   1/1     Running   7          162m

2. create  asb namespace and operator group:
$ oc create namespace openshift-ansible-service-broker
$ oc create -f asb-operatorgroup.yaml
[core@ip-10-0-8-250 asb]$ cat asb-operatorgroup.yaml 
apiVersion: operators.coreos.com/v1alpha2
kind: OperatorGroup
metadata:
  name: ansible-service-broker
  namespace: openshift-ansible-service-broker
spec:
  targetNamespaces:
    - openshift-ansible-service-broker

3. install asb operator to openshift-ansible-service-broker project
operator hub ->show community operaotrs->choose 'Automaiton Broker Operator' -> install -> choose 'ansible-service-broker' og as Target -> click subscribe.

4. create automationbroker CR
In openshift-ansible-service-broker project->Installed Operator->choose 'Automation Broker Operator'->click 'new' in Automation Broker ->Using default CR showed in console to create.

 CR:
apiVersion: osb.openshift.io/v1alpha1
kind: AutomationBroker
metadata:
  name: ansible-service-broker
  namespace: openshift-ansible-service-broker
spec:
  createBrokerNamespace: 'false'
  waitForBroker: 'false'


Actual results:
automation broker failed to created.
[core@ip-10-0-8-250 asb]$ oc get pod 
NAME                                               READY   STATUS    RESTARTS   AGE
automation-broker-operator-7998bfc9c7-dgp26        1/1     Running   0          90s
installed-community-ansible-service-broker-9qppd   1/1     Running   0          8m28s

[core@ip-10-0-8-250 asb]$ oc get automationbroker -o yaml
apiVersion: v1
items:
- apiVersion: osb.openshift.io/v1alpha1
  kind: AutomationBroker
  metadata:
    creationTimestamp: 2019-02-13T03:45:10Z
    finalizers:
    - finalizer.osb.openshift.io
    generation: 1
    name: ansible-service-broker
    namespace: openshift-ansible-service-broker
    resourceVersion: "667857"
    selfLink: /apis/osb.openshift.io/v1alpha1/namespaces/openshift-ansible-service-broker/automationbrokers/ansible-service-broker
    uid: c2e5b9d6-2f41-11e9-8c20-0abcc3a1d520
  spec:
    createBrokerNamespace: "false"
    waitForBroker: "false"
  status:
    conditions:
    - lastTransitionTime: 2019-02-13T03:45:10Z
      message: Running reconciliation
      reason: Running
      status: "False"
      type: Running
    - ansibleResult:
        changed: 0
        completion: 2019-02-13T03:45:12.24313
        failures: 1
        ok: 0
        skipped: 0
      lastTransitionTime: 2019-02-13T03:45:12Z
      message: |+
        An unhandled exception occurred while running the lookup plugin 'k8s'. Error was a <class 'kubernetes.client.rest.ApiException'>, original message: (401)
        Reason: Unauthorized
        HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 13 Feb 2019 03:45:12 GMT', 'Content-Length': '129', 'Content-Type': 'application/json', 'Cache-Control': 'no-store'})
        HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}

      reason: Failed
      status: "True"
      type: Failure
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""


Expected results:
automationbroker cr create successfully

Additional info:

Comment 1 Shawn Hurley 2019-02-13 16:03:56 UTC
Please note that to install the automation-broker you will need to take a single necessary action of creating the a cluster role binding.

an example is: 

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: "automation-broker-operator"
roleRef:
  name: admin
  kind: ClusterRole
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: automation-broker-operator
  namespace: openshift-ansible-service-broker

This should then complete the outcome.

We have a PR to add a better error message here: 
https://github.com/openshift/ansible-service-broker/pull/1182

I do not believe this should be a test blocker anymore or a beta 2 blocker but will use this BZ for the PR mentioned above.

Comment 2 Zihan Tang 2019-02-14 06:54:31 UTC
Tried again in 4.0.0-0.nightly-2019-02-13-204401
It's not always reproduced, remove TestBlocker

Comment 3 Zihan Tang 2019-02-27 09:33:46 UTC
(In reply to Shawn Hurley from comment #1)
> Please note that to install the automation-broker you will need to take a
> single necessary action of creating the a cluster role binding.
>.....
> We have a PR to add a better error message here: 
> https://github.com/openshift/ansible-service-broker/pull/1182
> 
> I do not believe this should be a test blocker anymore or a beta 2 blocker
> but will use this BZ for the PR mentioned above.

We now can install asb from marketplace. 
But still need to create clusterrolebinding before provision. Whether this is workaround or desigined?If this designed, we need to add this step in doc.

Comment 4 Alay Patel 2019-03-05 19:18:24 UTC
As per my conversation with @shurley, `creating clusterrolebinding before provision` is documented. Hence, I am closing the issue.

Comment 5 Shawn Hurley 2019-03-05 20:26:05 UTC
I would like to test the new error message.

Alay is correct that cluster role binding is being documented already.

Comment 7 Zihan Tang 2019-03-08 03:12:22 UTC
Verified,
Cluster version is 4.0.0-0.nightly-2019-03-05-065158
ASB: 1.4.4

create clusterrolebinding process is added to doc.

Comment 10 errata-xmlrpc 2019-06-04 10:44:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758