Bug 1467860 - system:serviceaccount:kube-service-catalog:default does not have enough permission
Summary: system:serviceaccount:kube-service-catalog:default does not have enough permi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: ewolinet
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-07-05 11:05 UTC by Johnny Liu
Modified: 2017-08-16 19:51 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
This is a bug that originated from a new 3.6 feature
Clone Of:
Environment:
Last Closed: 2017-08-10 05:29:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1716 0 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.6 RPM Release Advisory 2017-08-10 09:02:50 UTC

Description Johnny Liu 2017-07-05 11:05:37 UTC
Description of problem:
see the following details.

Version-Release number of selected component (if applicable):
openshift v3.6.126.8
kubernetes v1.6.1+5115d708d7
etcd 3.2.0
openshift-ansible-roles-3.6.132-1.git.0.0d0f54a.el7.noarch

How reproducible:
Always


Steps to Reproduce:
1. enable service catalog deployment in inventory host file
2. after installation, checking catalog controller manager pod log
# oc get po -n kube-service-catalog
NAME                       READY     STATUS    RESTARTS   AGE
apiserver-nsqrz            1/1       Running   0          1d
controller-manager-qlnq1   1/1       Running   0          1h
# oc logs -f controller-manager-qlnq1 -n kube-service-catalog

3.

Actual results:
<--snip-->
I0705 08:49:20.320088       1 reflector.go:236] Listing and watching *v1alpha1.Binding from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
I0705 08:49:20.321700       1 reflector.go:236] Listing and watching *v1alpha1.Instance from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
E0705 08:49:20.329031       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Instance: User "system:serviceaccount:kube-service-catalog:default" cannot list all instances.servicecatalog.k8s.io in the cluster
E0705 08:49:20.329075       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Binding: User "system:serviceaccount:kube-service-catalog:default" cannot list all bindings.servicecatalog.k8s.io in the cluster
I0705 08:49:21.254071       1 reflector.go:236] Listing and watching *v1alpha1.Broker from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
E0705 08:49:21.259123       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Broker: User "system:serviceaccount:kube-service-catalog:default" cannot list all brokers.servicecatalog.k8s.io in the cluster
I0705 08:49:21.329252       1 reflector.go:236] Listing and watching *v1alpha1.Instance from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
I0705 08:49:21.330640       1 reflector.go:236] Listing and watching *v1alpha1.Binding from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
E0705 08:49:21.338100       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Instance: User "system:serviceaccount:kube-service-catalog:default" cannot list all instances.servicecatalog.k8s.io in the cluster
E0705 08:49:21.338244       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Binding: User "system:serviceaccount:kube-service-catalog:default" cannot list all bindings.servicecatalog.k8s.io in the cluster
I0705 08:49:21.660393       1 leaderelection.go:204] succesfully renewed lease kube-service-catalog/service-catalog-controller-manager
I0705 08:49:22.259340       1 reflector.go:236] Listing and watching *v1alpha1.Broker from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
E0705 08:49:22.264672       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Broker: User "system:serviceaccount:kube-service-catalog:default" cannot list all brokers.servicecatalog.k8s.io in the cluster
I0705 08:49:22.338319       1 reflector.go:236] Listing and watching *v1alpha1.Instance from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
I0705 08:49:22.340350       1 reflector.go:236] Listing and watching *v1alpha1.Binding from github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61
E0705 08:49:22.347762       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Instance: User "system:serviceaccount:kube-service-catalog:default" cannot list all instances.servicecatalog.k8s.io in the cluster
E0705 08:49:22.347967       1 reflector.go:201] github.com/kubernetes-incubator/service-catalog/pkg/client/informers_generated/externalversions/factory.go:61: Failed to list *v1alpha1.Binding: User "system:serviceaccount:kube-service-catalog:default" cannot list all bindings.servicecatalog.k8s.io in the cluster
<--snip-->

after adding "cluster-admin" role to "system:serviceaccount:kube-service-catalog:default", no error is seen.

Expected results:
No error is seen in service catalog pod.

Additional info:

Comment 1 ewolinet 2017-07-05 21:04:03 UTC
https://github.com/openshift/openshift-ansible/pull/4684 fixes this

Comment 3 Johnny Liu 2017-07-07 11:25:20 UTC
Currently the latest puddle QE get is AtomicOpenShift/3.6/2017-07-07.2, but its openshift-ansible version openshift-ansible-3.6.126.14-1.git.0.efd80ab.el7, does not have this PR merged.

Comment 5 Johnny Liu 2017-07-10 07:09:30 UTC
Verified this bug with openshift-ansible-3.6.138-1.git.0.2c647a9.el7.noarch, and PASS.


No permission error log is seen in controller-manager pod.

Comment 7 Johnny Liu 2017-07-11 10:37:18 UTC
Pls ignore comment 5, it is tested with wrong steps.

Re-test this bug with openshift-ansible-3.6.140-1.git.0.4a02427.el7.noarch, and FAIL.

# oc get broker ansible-service-broker -o json
{
    "apiVersion": "servicecatalog.k8s.io/v1alpha1",
    "kind": "Broker",
    "metadata": {
        "creationTimestamp": "2017-07-11T10:27:10Z",
        "finalizers": [
            "kubernetes-incubator/service-catalog"
        ],
        "name": "ansible-service-broker",
        "resourceVersion": "3643",
        "selfLink": "/apis/servicecatalog.k8s.io/v1alpha1/brokersansible-service-broker",
        "uid": "7fa0a1bd-6623-11e7-8be8-0a580a800007"
    },
    "spec": {
        "url": "http://asb.openshift-ansible-service-broker.svc:1338"
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2017-07-11T10:27:10Z",
                "message": "Error syncing catalog from Broker. Error reconciling serviceClass \"postgresql-apb\" (broker \"ansible-service-broker\"): User \"system:serviceaccount:kube-service-catalog:service-catalog-controller\" cannot create serviceclasses.servicecatalog.k8s.io at the cluster scope",
                "reason": "ErrorSyncingCatalog",
                "status": "False",
                "type": "Ready"
            }
        ]
    }
}


Log from service catalog conroller-manager pod:
<--snip-->
I0711 10:28:33.491522       1 controller_broker.go:196] Successfully fetched 2 catalog entries for Broker ansible-service-broker
I0711 10:28:33.491533       1 controller_broker.go:198] Converting catalog response for Broker ansible-service-broker into service-catalog API
I0711 10:28:33.491677       1 controller_broker.go:207] Successfully converted catalog payload from Broker ansible-service-broker to service-catalog API
I0711 10:28:33.491684       1 controller_broker.go:218] Reconciling serviceClass postgresql-apb (broker ansible-service-broker)
E0711 10:28:33.503013       1 controller_broker.go:319] Error creating serviceClass postgresql-apb from Broker ansible-service-broker: User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot create serviceclasses.servicecatalog.k8s.io at the cluster scope
W0711 10:28:33.503031       1 controller_broker.go:226] Error reconciling serviceClass "postgresql-apb" (broker "ansible-service-broker"): User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot create serviceclasses.servicecatalog.k8s.io at the cluster scope
I0711 10:28:33.503052       1 controller_broker.go:403] Updating ready condition for Broker ansible-service-broker to False
I0711 10:28:33.503495       1 event.go:217] Event(v1.ObjectReference{Kind:"Broker", Namespace:"", Name:"ansible-service-broker", UID:"7fa0a1bd-6623-11e7-8be8-0a580a800007", APIVersion:"servicecatalog.k8s.io", ResourceVersion:"3643", FieldPath:""}): type: 'Warning' reason: 'ErrorSyncingCatalog' Error reconciling serviceClass "postgresql-apb" (broker "ansible-service-broker"): User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot create serviceclasses.servicecatalog.k8s.io at the cluster scope
I0711 10:28:33.521721       1 controller_broker.go:408] Updated ready condition for Broker ansible-service-broker to False
I0711 10:28:33.521751       1 controller.go:195] Error syncing Broker ansible-service-broker: User "system:serviceaccount:kube-service-catalog:service-catalog-controller" cannot create serviceclasses.servicecatalog.k8s.io at the cluster scope
I0711 10:28:34.943007       1 leaderelection.go:204] succesfully renewed lease kube-service-catalog/service-catalog-controller-manager
I0711 10:28:36.951757       1 leaderelection.go:204] succesfully renewed lease kube-service-catalog/service-catalog-controller-manager
I0711 10:28:38.960979       1 leaderelection.go:204] succesfully renewed lease kube-service-catalog/service-catalog-controller-manager
I0711 10:28:40.969358       1 leaderelection.go:204] succesfully renewed lease kube-service-catalog/service-catalog-controller-manager
<--snip-->

Comment 10 Johnny Liu 2017-07-14 10:21:23 UTC
Verified this bug with openshift-ansible-3.6.144-1.git.0.50e12bf.el7.noarch, and PASS.


After run "curl -H 'X-Broker-API-Version: 2.9' -X POST http://<asb-svc>:1338/v2/bootstrap".

# oc get broker ansible-service-broker -o json
{
    "apiVersion": "servicecatalog.k8s.io/v1alpha1",
    "kind": "Broker",
    "metadata": {
        "creationTimestamp": "2017-07-14T09:00:12Z",
        "finalizers": [
            "kubernetes-incubator/service-catalog"
        ],
        "name": "ansible-service-broker",
        "resourceVersion": "11168",
        "selfLink": "/apis/servicecatalog.k8s.io/v1alpha1/brokersansible-service-broker",
        "uid": "d8a4314b-6872-11e7-9648-0a580a020002"
    },
    "spec": {
        "url": "http://asb.openshift-ansible-service-broker.svc:1338"
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2017-07-14T10:11:52Z",
                "message": "Successfully fetched catalog entries from broker.",
                "reason": "FetchedCatalog",
                "status": "True",
                "type": "Ready"
            }
        ]
    }
}

No permission error is seen in manager-controller pod.

Comment 12 errata-xmlrpc 2017-08-10 05:29:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1716


Note You need to log in before you can comment on or make changes to this bug.