Bug 1747376 - ASB and TSB operator pod create failed with "Error: container create failed: container_linux.go:345: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory"
Summary: ASB and TSB operator pod create failed with "Error: container create failed: ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Service Broker
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.2.0
Assignee: Fabian von Feilitzsch
QA Contact: Cuiping HUO
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-08-30 09:12 UTC by Cuiping HUO
Modified: 2019-10-16 06:39 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-16 06:39:13 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:2922 0 None None None 2019-10-16 06:39:23 UTC

Description Cuiping HUO 2019-08-30 09:12:21 UTC
Description of problem:
ASB and TSB operator pod create failed with "Error: container create failed: container_linux.go:345: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory"

Version-Release number of selected component (if applicable):

cluster version: 4.2.0-0.nightly-2019-08-28-152644
ASB operator commit.id:0050a51a8cd7f635615d9b1a5d08916aa5451d5e
TSB operator commit.id:6662db1ea3b0150d98775db608359c14bd78f669

How reproducible:
Always

Steps to Reproduce:
1.get ASB and TSB v4.2.0 packagemanifest by using qe downstram image appregistry:aosqe42
2.install ASB and TSB operator through web console
3.exec `sudo docker run -it IMAGE_ID" to check image

Actual results:
asb and tsb operator install failed for pod can not be created "Error: container create failed: container_linux.go:345: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory"
`sudo docker run -it IMAGE_ID` with "Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory".

Expected results:
asb and tsb operator shoud be installed successfully
`sudo docker run -it IMAGE_ID` should success.


Additional info:
$ sudo docker images
REPOSITORY                                                                                            TAG                    IMAGE ID            CREATED             SIZE
brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift/ose-ansible-service-broker-operator    v4.2.0                 054b3beb3802        33 hours ago        610 MB
brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift/ose-template-service-broker-operator   v4.2.0                 b8a1d7558620        37 hours ago        610 MB

$ sudo docker run -it 054b3beb3802 
/usr/bin/docker-current: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory".

$ sudo docker run -it b8a1d7558620 
/usr/bin/docker-current: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory".



$ oc get packagemanifest openshiftansibleservicebroker -o yaml
status:
  catalogSource: qe-app-registry
  catalogSourceDisplayName: ""
  catalogSourceNamespace: openshift-marketplace
  catalogSourcePublisher: ""
  channels:
  - currentCSV: openshiftansibleservicebroker.4.2.0
$ oc get packagemanifest openshiftansibleservicebroker -o yaml
status:
  catalogSource: qe-app-registry
  catalogSourceDisplayName: ""
  catalogSourceNamespace: openshift-marketplace
  catalogSourcePublisher: ""
  channels:
  - currentCSV: openshifttemplateservicebrokeroperator.4.2.0

$ oc get po -n openshift-ansible-service-broker
NAME                                                        READY   STATUS                 RESTARTS   AGE
openshift-ansible-service-broker-operator-db65bc7b6-hx8fr   0/1     CreateContainerError   0          14m
[chuo@dhcp-140-51 .kube]$ oc describe po openshift-ansible-service-broker-operator-db65bc7b6-hx8fr -n openshift-ansible-service-broker
Name:               openshift-ansible-service-broker-operator-db65bc7b6-hx8fr
Namespace:          openshift-ansible-service-broker
Priority:           0
PriorityClassName:  <none>
Node:               qe-minmli-share-0830-gpkqj-worker-centralus2-ltnhl/10.0.32.6
Start Time:         Fri, 30 Aug 2019 16:22:01 +0800
Labels:             name=openshift-ansible-service-broker-operator-alm-owned
                    pod-template-hash=db65bc7b6
Annotations:        alm-examples:
                      [{"apiVersion":"osb.openshift.io/v1", "kind":"AutomationBroker", "metadata":{"name":"ansible-service-broker","namespace":"ansible-service-...
                    capabilities: Seamless Upgrades
                    containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0
                    description:
                      OpenShift Ansible Service Broker is an implementation of the [Open Service Broker API](https://github.com/openservicebrokerapi/servicebrok...
                    olm.operatorGroup: openshift-ansible-service-broker-r59px
                    olm.operatorNamespace: openshift-ansible-service-broker
                    olm.skipRange: >=4.1.0 <4.2.0
                    olm.targetNamespaces: openshift-ansible-service-broker
                    openshift.io/scc: restricted
Status:             Pending
IP:                 10.129.2.34
Controlled By:      ReplicaSet/openshift-ansible-service-broker-operator-db65bc7b6
Containers:
  openshift-ansible-service-broker-operator:
    Container ID:   
    Image:          image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CreateContainerError
    Ready:          False
    Restart Count:  0
    Environment:
      IMAGE:            image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0
      OPERATOR_NAME:    openshift-ansible-service-broker-operator
      POD_NAME:         openshift-ansible-service-broker-operator-db65bc7b6-hx8fr (v1:metadata.name)
      WATCH_NAMESPACE:  openshift-ansible-service-broker (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from openshift-ansible-service-broker-operator-token-rzzj7 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  openshift-ansible-service-broker-operator-token-rzzj7:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  openshift-ansible-service-broker-operator-token-rzzj7
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                   From                                                         Message
  ----     ------     ----                  ----                                                         -------
  Normal   Scheduled  14m                   default-scheduler                                            Successfully assigned openshift-ansible-service-broker/openshift-ansible-service-broker-operator-db65bc7b6-hx8fr to qe-minmli-share-0830-gpkqj-worker-centralus2-ltnhl
  Normal   Pulling    14m                   kubelet, qe-minmli-share-0830-gpkqj-worker-centralus2-ltnhl  Pulling image "image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0"
  Normal   Pulled     13m                   kubelet, qe-minmli-share-0830-gpkqj-worker-centralus2-ltnhl  Successfully pulled image "image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0"
  Warning  Failed     11m (x12 over 13m)    kubelet, qe-minmli-share-0830-gpkqj-worker-centralus2-ltnhl  Error: container create failed: container_linux.go:345: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory"
  Normal   Pulled     4m33s (x41 over 13m)  kubelet, qe-minmli-share-0830-gpkqj-worker-centralus2-ltnhl  Container image "image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0" already present on machine

$ oc get po -n openshift-template-service-broker
NAME                                                          READY   STATUS                 RESTARTS   AGE
openshift-template-service-broker-operator-7895df966f-xh967   0/1     CreateContainerError   0          13s
[chuo@dhcp-140-51 .kube]$ oc describe po openshift-template-service-broker-operator-7895df966f-xh967 -n openshift-template-service-broker
Name:               openshift-template-service-broker-operator-7895df966f-xh967
Namespace:          openshift-template-service-broker
Priority:           0
PriorityClassName:  <none>
Node:               qe-minmli-share-0830-gpkqj-worker-centralus3-znk2h/10.0.32.5
Start Time:         Fri, 30 Aug 2019 16:34:44 +0800
Labels:             name=openshift-template-service-broker-operator-alm-owned
                    pod-template-hash=7895df966f
Annotations:        alm-examples:
                      [{"apiVersion":"osb.openshift.io/v1","kind":"TemplateServiceBroker","metadata":{"name":"template-service-broker","namespace":"template-ser...
                    capabilities: Seamless Upgrades
                    containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker:v4.2.0
                    description:
                      OpenShift Template Service Broker is an implementation of the [Open Service Broker API]https://github.com/openservicebrokerapi/servicebrok...
                    olm.operatorGroup: openshift-template-service-broker-frvvd
                    olm.operatorNamespace: openshift-template-service-broker
                    olm.skipRange: >=4.1.0 <4.2.0
                    olm.targetNamespaces: openshift-template-service-broker
                    openshift.io/scc: restricted
Status:             Pending
IP:                 10.131.0.50
Controlled By:      ReplicaSet/openshift-template-service-broker-operator-7895df966f
Containers:
  openshift-template-service-broker-operator:
    Container ID:   
    Image:          image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker-operator:v4.2.0
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CreateContainerError
    Ready:          False
    Restart Count:  0
    Environment:
      IMAGE:            image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker:v4.2.0
      OPERATOR_NAME:    openshift-template-service-broker-operator
      POD_NAME:         openshift-template-service-broker-operator-7895df966f-xh967 (v1:metadata.name)
      WATCH_NAMESPACE:  openshift-template-service-broker (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from openshift-template-service-broker-operator-token-2qdbf (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  openshift-template-service-broker-operator-token-2qdbf:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  openshift-template-service-broker-operator-token-2qdbf
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age               From                                                         Message
  ----     ------     ----              ----                                                         -------
  Normal   Scheduled  24s               default-scheduler                                            Successfully assigned openshift-template-service-broker/openshift-template-service-broker-operator-7895df966f-xh967 to qe-minmli-share-0830-gpkqj-worker-centralus3-znk2h
  Normal   Pulling    21s               kubelet, qe-minmli-share-0830-gpkqj-worker-centralus3-znk2h  Pulling image "image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker-operator:v4.2.0"
  Normal   Pulled     19s               kubelet, qe-minmli-share-0830-gpkqj-worker-centralus3-znk2h  Successfully pulled image "image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker-operator:v4.2.0"
  Warning  Failed     4s (x3 over 18s)  kubelet, qe-minmli-share-0830-gpkqj-worker-centralus3-znk2h  Error: container create failed: container_linux.go:345: starting container process caused "exec: \"/tini\": stat /tini: no such file or directory"
  Normal   Pulled     4s (x2 over 18s)  kubelet, qe-minmli-share-0830-gpkqj-worker-centralus3-znk2h  Container image "image-registry.openshift-image-registry.svc:5000/openshift/ose-template-service-broker-operator:v4.2.0" already present on machine

Comment 1 Fabian von Feilitzsch 2019-08-30 12:27:48 UTC
This was an oversight when updating the downstream image to match the upstream, fix posted here which uses the proper path for the tini binary when it's installed via RPM: https://github.com/openshift/ocp-release-operator-sdk/pull/21

Comment 3 Cuiping HUO 2019-09-06 03:26:55 UTC
Verify failed.
cluster version:4.2.0-0.nightly-2019-09-04-142146

asb and tsb operator pod status "CrashLoopBackOff"

ASB:
$ oc get po -n openshift-ansible-service-broker
NAME                                                        READY   STATUS             RESTARTS   AGE
openshift-ansible-service-broker-operator-77c65b75f-zp8qj   0/1     CrashLoopBackOff   1          18s

$ oc describe po openshift-ansible-service-broker-operator-77c65b75f-zp8qj -n openshift-ansible-service-broker
Name:               openshift-ansible-service-broker-operator-77c65b75f-zp8qj
Namespace:          openshift-ansible-service-broker
Priority:           0
PriorityClassName:  <none>
Node:               ip-10-0-151-13.us-east-2.compute.internal/10.0.151.13
Start Time:         Fri, 06 Sep 2019 11:12:39 +0800
Labels:             name=openshift-ansible-service-broker-operator-alm-owned
                    pod-template-hash=77c65b75f
Annotations:        alm-examples:
                      [{"apiVersion":"osb.openshift.io/v1", "kind":"AutomationBroker", "metadata":{"name":"ansible-service-broker","namespace":"ansible-service-...
                    capabilities: Seamless Upgrades
                    containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0
                    description:
                      OpenShift Ansible Service Broker is an implementation of the [Open Service Broker API](https://github.com/openservicebrokerapi/servicebrok...
                    olm.operatorGroup: openshift-ansible-service-broker-ctkcx
                    olm.operatorNamespace: openshift-ansible-service-broker
                    olm.skipRange: >=4.1.0 <4.2.0
                    olm.targetNamespaces: openshift-ansible-service-broker
                    openshift.io/scc: restricted
Status:             Running
IP:                 10.128.2.18
Controlled By:      ReplicaSet/openshift-ansible-service-broker-operator-77c65b75f
Containers:
  openshift-ansible-service-broker-operator:
    Container ID:   cri-o://efb165acc2b1fb760654074be085cafe87dcaf61743cca4972a7941f2d451677
    Image:          image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0
    Image ID:       image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator@sha256:8b3da84b5bd55327c3fd6b0cba1d2a4d6f6e001ccc13d30dfeb5906ee82177d0
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Fri, 06 Sep 2019 11:12:57 +0800
      Finished:     Fri, 06 Sep 2019 11:12:57 +0800
    Ready:          False
    Restart Count:  2
    Environment:
      IMAGE:            image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0
      OPERATOR_NAME:    openshift-ansible-service-broker-operator
      POD_NAME:         openshift-ansible-service-broker-operator-77c65b75f-zp8qj (v1:metadata.name)
      WATCH_NAMESPACE:  openshift-ansible-service-broker (v1:metadata.namespace)
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from openshift-ansible-service-broker-operator-token-gkns5 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  openshift-ansible-service-broker-operator-token-gkns5:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  openshift-ansible-service-broker-operator-token-gkns5
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From                                                Message
  ----     ------     ----               ----                                                -------
  Normal   Scheduled  41s                default-scheduler                                   Successfully assigned openshift-ansible-service-broker/openshift-ansible-service-broker-operator-77c65b75f-zp8qj to ip-10-0-151-13.us-east-2.compute.internal
  Normal   Pulled     23s (x3 over 38s)  kubelet, ip-10-0-151-13.us-east-2.compute.internal  Container image "image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0" already present on machine
  Normal   Created    23s (x3 over 38s)  kubelet, ip-10-0-151-13.us-east-2.compute.internal  Created container openshift-ansible-service-broker-operator
  Normal   Started    23s (x3 over 38s)  kubelet, ip-10-0-151-13.us-east-2.compute.internal  Started container openshift-ansible-service-broker-operator
  Warning  BackOff    8s (x4 over 36s)   kubelet, ip-10-0-151-13.us-east-2.compute.internal  Back-off restarting failed container
[chuo@dhcp-140-51 .kube]$ oc logs -f openshift-ansible-service-broker-operator-77c65b75f-zp8qj -n openshift-ansible-service-broker 
{"level":"info","ts":1567739603.4180648,"logger":"cmd","msg":"Go Version: go1.12.8"}
{"level":"info","ts":1567739603.4181044,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1567739603.4181125,"logger":"cmd","msg":"Version of operator-sdk: v0.10.0+git"}
{"level":"info","ts":1567739603.4181318,"logger":"cmd","msg":"Watching namespace.","Namespace":"openshift-ansible-service-broker"}
{"level":"info","ts":1567739603.582024,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"osb.openshift.io","Options.Version":"v1","Options.Kind":"AutomationBroker"}
{"level":"info","ts":1567739603.5822206,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"automationbroker-controller","source":"kind source: osb.openshift.io/v1, Kind=AutomationBroker"}
{"level":"info","ts":1567739603.5823593,"logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":1567739603.742106,"logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
{"level":"info","ts":1567739603.74214,"logger":"leader","msg":"Continuing as the leader."}
{"level":"error","ts":1567739603.925735,"logger":"cmd","msg":"Exposing metrics port failed.","Namespace":"openshift-ansible-service-broker","error":"failed to create or get service for metrics: services \"openshift-ansible-service-broker-operator-metrics\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible.Run\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/run.go:142\ngithub.com/operator-framework/operator-sdk/cmd/operator-sdk/run.newRunAnsibleCmd.func1\n\tsrc/github.com/operator-framework/operator-sdk/cmd/operator-sdk/run/ansible.go:38\ngithub.com/spf13/cobra.(*Command).execute\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/spf13/cobra/command.go:762\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/spf13/cobra/command.go:852\ngithub.com/spf13/cobra.(*Command).Execute\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/spf13/cobra/command.go:800\nmain.main\n\tsrc/github.com/operator-framework/operator-sdk/cmd/operator-sdk/main.go:85\nruntime.main\n\t/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/runtime/proc.go:200"}
Error: failed to create or get service for metrics: services "openshift-ansible-service-broker-operator-metrics" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>
Usage:
  operator-sdk run ansible [flags]

Flags:
  -h, --help                             help for ansible
      --inject-owner-ref                 The ansible operator will inject owner references unless this flag is false (default true)
      --max-workers int                  Maximum number of workers to use. Overridden by environment variable. (default 1)
      --reconcile-period duration        Default reconcile period for controllers (default 1m0s)
      --watches-file string              Path to the watches file to use (default "./watches.yaml")
      --zap-devel                        Enable zap development mode (changes defaults to console encoder, debug log level, and disables sampling)
      --zap-encoder encoder              Zap log encoding ('json' or 'console')
      --zap-level level                  Zap log level (one of 'debug', 'info', 'error' or any integer value > 0) (default info)
      --zap-sample sample                Enable zap log sampling. Sampling will be disabled for integer log levels > 1
      --zap-time-encoding timeEncoding   Sets the zap time format ('epoch', 'millis', 'nano', or 'iso8601') (default )

Global Flags:
      --verbose   Enable verbose logging

TSB:
$ oc get po -n openshift-template-service-broker
NAME                                                          READY   STATUS   RESTARTS   AGE
openshift-template-service-broker-operator-76bf957946-fmldr   0/1     Error    1          27s

$ oc get po -n openshift-template-service-broker
NAME                                                          READY   STATUS   RESTARTS   AGE
openshift-template-service-broker-operator-76bf957946-fmldr   0/1     Error    1          29s
[chuo@dhcp-140-51 .kube]$ oc logs -f openshift-template-service-broker-operator-76bf957946-fmldr -n openshift-template-service-broker
{"level":"info","ts":1567740289.5560179,"logger":"cmd","msg":"Go Version: go1.12.8"}
{"level":"info","ts":1567740289.5565288,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1567740289.5565383,"logger":"cmd","msg":"Version of operator-sdk: v0.10.0+git"}
{"level":"info","ts":1567740289.556569,"logger":"cmd","msg":"Watching namespace.","Namespace":"openshift-template-service-broker"}
{"level":"info","ts":1567740289.7324212,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"osb.openshift.io","Options.Version":"v1","Options.Kind":"TemplateServiceBroker"}
{"level":"info","ts":1567740289.7326264,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"templateservicebroker-controller","source":"kind source: osb.openshift.io/v1, Kind=TemplateServiceBroker"}
{"level":"info","ts":1567740289.7327716,"logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":1567740289.897709,"logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
{"level":"info","ts":1567740289.8977442,"logger":"leader","msg":"Continuing as the leader."}
{"level":"error","ts":1567740290.049456,"logger":"cmd","msg":"Exposing metrics port failed.","Namespace":"openshift-template-service-broker","error":"failed to initialize service object for metrics: replicasets.apps \"openshift-template-service-broker-operator-76bf957946\" is forbidden: User \"system:serviceaccount:openshift-template-service-broker:openshift-template-service-broker-operator\" cannot get resource \"replicasets\" in API group \"apps\" in the namespace \"openshift-template-service-broker\"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible.Run\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/run.go:142\ngithub.com/operator-framework/operator-sdk/cmd/operator-sdk/run.newRunAnsibleCmd.func1\n\tsrc/github.com/operator-framework/operator-sdk/cmd/operator-sdk/run/ansible.go:38\ngithub.com/spf13/cobra.(*Command).execute\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/spf13/cobra/command.go:762\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/spf13/cobra/command.go:852\ngithub.com/spf13/cobra.(*Command).Execute\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/spf13/cobra/command.go:800\nmain.main\n\tsrc/github.com/operator-framework/operator-sdk/cmd/operator-sdk/main.go:85\nruntime.main\n\t/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/runtime/proc.go:200"}
Error: failed to initialize service object for metrics: replicasets.apps "openshift-template-service-broker-operator-76bf957946" is forbidden: User "system:serviceaccount:openshift-template-service-broker:openshift-template-service-broker-operator" cannot get resource "replicasets" in API group "apps" in the namespace "openshift-template-service-broker"
Usage:
  operator-sdk run ansible [flags]

Flags:
  -h, --help                             help for ansible
      --inject-owner-ref                 The ansible operator will inject owner references unless this flag is false (default true)
      --max-workers int                  Maximum number of workers to use. Overridden by environment variable. (default 1)
      --reconcile-period duration        Default reconcile period for controllers (default 1m0s)
      --watches-file string              Path to the watches file to use (default "./watches.yaml")
      --zap-devel                        Enable zap development mode (changes defaults to console encoder, debug log level, and disables sampling)
      --zap-encoder encoder              Zap log encoding ('json' or 'console')
      --zap-level level                  Zap log level (one of 'debug', 'info', 'error' or any integer value > 0) (default info)
      --zap-sample sample                Enable zap log sampling. Sampling will be disabled for integer log levels > 1
      --zap-time-encoding timeEncoding   Sets the zap time format ('epoch', 'millis', 'nano', or 'iso8601') (default )

Global Flags:
      --verbose   Enable verbose logging

Comment 5 Fabian von Feilitzsch 2019-09-06 18:21:57 UTC
Hmm, I think that's an unrelated issue.

It seems that the CSVs may not be up to date. As far as I can tell the CSVs provide the required permissions, so I'm not sure why they'd be failing like this. Do you know how the downstream appregistry is populated, and can we inspect the CSVs there?

TSBO permission: https://github.com/openshift/template-service-broker-operator/blob/release-4.2/deploy/olm-catalog/openshift-template-service-broker-manifests/4.2/openshifttemplateservicebrokeroperator.v4.2.0.clusterserviceversion.yaml#L95
ASBO permission: https://github.com/openshift/ansible-service-broker/blob/release-4.2/operator/deploy/olm-catalog/openshift-ansible-service-broker-manifests/4.2/openshiftansibleservicebroker.v4.2.0.clusterserviceversion.yaml#L74

Comment 6 Jesus M. Rodriguez 2019-09-06 21:30:41 UTC
See comment #5 for the needinfo

Comment 9 Cuiping HUO 2019-09-09 13:01:54 UTC
@Fabian,
Thanks for your information of CSVs. I update the latest ASB and TSB CSVs. And find TSB works. But ASB CSV is under status "PENDING",because of "Service account does not exist" of "openshift-ansible-service-broker-operator".

3 changes of upstreams ASB CSV(https://github.com/openshift/ansible-service-broker/blob/release-4.2/operator/deploy/olm-catalog/openshift-ansible-service-broker-manifests/4.2/openshiftansibleservicebroker.v4.2.0.clusterserviceversion.yaml) changes shows as below:
1. .metadata.containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0
2. .spec.deployments.spec.containers.image: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0
3. .spec.deployments.spec.containers.env:
                - name: IMAGE
                  value: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0

$ oc get sa -n openshift-ansible-service-broker
NAME       SECRETS   AGE
builder    2         152m
default    2         152m
deployer   2         152m

$ oc get csv openshiftansibleservicebroker.v4.2.0 -n openshift-ansible-service-broker -o yaml
apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  annotations:
    alm-examples: '[{"apiVersion":"osb.openshift.io/v1", "kind":"AutomationBroker",
      "metadata":{"name":"ansible-service-broker","namespace":"ansible-service-broker"},
      "spec":{"createBrokerNamespace":"false","waitForBroker":"false", "registries":
      [{"type": "rhcc", "name": "rhcc", "url": "https://registry.redhat.io", "white_list":
      [".*-apb$"], "auth_type": "secret", "auth_name": "asb-registry-auth"}]}}]'
    capabilities: Seamless Upgrades
    containerImage: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0
    description: OpenShift Ansible Service Broker is an implementation of the [Open
      Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
    olm.operatorGroup: openshift-ansible-service-broker-8vxps
    olm.operatorNamespace: openshift-ansible-service-broker
    olm.targetNamespaces: openshift-ansible-service-broker
  creationTimestamp: 2019-09-09T12:44:16Z
  generation: 1
  labels:
    olm.api.2fc6c4e2d95a77f0: provided
    olm.api.880c523f5d0ea900: provided
    olm.api.7044497f8b7e10a9: provided
    olm.api.dba600531489ce6c: provided
  name: openshiftansibleservicebroker.v4.2.0
  namespace: openshift-ansible-service-broker
  resourceVersion: "202061"
  selfLink: /apis/operators.coreos.com/v1alpha1/namespaces/openshift-ansible-service-broker/clusterserviceversions/openshiftansibleservicebroker.v4.2.0
  uid: 88a5a982-d2ff-11e9-bec4-0a5191f3ea26
spec:
  apiservicedefinitions: {}
  customresourcedefinitions:
    owned:
    - description: An Open Service Broker supporting management of application bundles
      displayName: Automation Broker
      kind: AutomationBroker
      name: automationbrokers.osb.openshift.io
      version: v1
    - description: An application bundle available for deployment via Automation Broker
      displayName: Automation Broker Bundle
      kind: Bundle
      name: bundles.automationbroker.io
      version: v1alpha1
    - description: An application bundle binding
      displayName: Automation Broker Bundle Binding
      kind: BundleBinding
      name: bundlebindings.automationbroker.io
      version: v1alpha1
    - description: An instance of an application bundle
      displayName: Automation Broker Bundle Instance
      kind: BundleInstance
      name: bundleinstances.automationbroker.io
      version: v1alpha1
  description: |
    OpenShift Ansible Service Broker is an implementation of the [Open Service Broker API](https://github.com/openservicebrokerapi/servicebroker)
    that manages applications defined in [Ansible Playbook Bundles](https://github.com/ansibleplaybookbundle/ansible-playbook-bundle).
    Ansible Playbook Bundles (APB) are a method of defining applications via a collection of Ansible Playbooks built into a container
    with an Ansible runtime with the playbooks corresponding to a type of request specified in the
    [Open Service Broker API Specification](https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#api-overview).

    Check out the [Keynote Demo from Red Hat Summit 2017](https://youtu.be/8MCbJmZQM9c?list=PLEGSLwUsxfEh4TE2GDU4oygCB-tmShkSn&t=4732)
  displayName: OpenShift Ansible Service Broker Operator
  icon:
  - base64data: iVBORw0KGgoAAAANSUhEUgAAATAAAAEwCAYAAAAw+y3zAAAACXBIWXMAAC4jAAAuIwF4pT92AAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAQJlJREFUeNrsfQtwVtW59kq4JAEhAY0IKISoPwbUhFr6S73Fo6WjHiSt48FWi9GZUtupBYc5rePpCDqtQ3uGAUqnx9oZG6i2pf5WwKqnnDrGS4tTagkiQQ81JKBQASEhGEIQ8u9nZ+/w5cu+rLX3Wnuvvff7zHwTLl++b9/Ws573XtDb28sIBAIhiSikS0AgEIjACAQCgQiMQCAQiMAIBAIRGIFAIBCBEQgEAhEYgUAgEIERCAQiMAKBQCACIxAIBCIwAoFABEYgEAhEYAQCgSAfQ+kSEAjRYHhxSY3xo8x4VVgvN7Qbryb8oaf7eCNdOXcUUDsdAkEZWdVaL5BVdYiP67AIDS8QWpNBbK10lYnACASZpFVv/KizSKtU8de1Ga/1IDSDzNYTgREIhKDEBcJqMF6TYzqEDovMVhpk1kQERiAQeMkLius5jQ5pm0VkDURgBALBi7wqWJ9fqlTDw4OJuTTtREZpFARCcCzSlLyYZc7+0iDZVss3RwqMQCAMUGCtLD6/lyheBeGmzUdGCoxACKdykoLrjNdWg3SXkgIjEAhQYEldPHD016dBjZECIxCyByTVNloRVCIwAiGj2JbgY0fw4bmkm5REYARCcDSk4ByWGCSW2PMgHxiBEBDGwkdhdivTN5VCBGt6uo/XkwIjEDICY8Gja0R9Sk7n7iQqMSIwAiEciaEG8R7WV49IJEYmJIGQSHOygvVl5iOyZ+eHoZwHqQowNa8jc5IIjEBIIrmhN9jWhB32PUmooyQCIxCiIbHGhKkw4HrdO8ISgREI0ZmYunaucAP8ehVWsEJLkBOfQIgAVgvoWsbn7EdQ4FUNDhtkq7UZSQRGIERHYlBgUGKrXIhsg/GaYfmeFjE9IptzDfW4iExIAoGQb1baU4ocpw9Z/480jbi7XmhrShKBEQh6k1yZpcbq84iszTLvQHBRRDi1TK0gAiMQkk9yUG9RRDi1i0qSD4xASD6i8pct1e3EicAIhITDCg5EQWLXWSPkiMAIBIJUEmtgfWkabukXILe2tKkw8oERtERuhC4HPLs/1EhutKw9a8NeraTZupzrh/NvtP7cysIn02rjCyMCI8RJTrV5xISFpzJloMNazCxnQTdmieQkDeLVJiJJBEZQrQRqcl74e7XGh9xmKZRGi+iarAz6tN0XpF7MDfkxY3TICyMCI8hcGLWWmqq1CCsNnUo7ckywRovU2hN+nypY+LrMB4zrsJIIjJAWwrouQ6e+zSY0q6FhEu/dUuPHkjDXwDj3GiIwQtJ2bpBVnQQTJE3YYBHa+qSYnJL6+U+J+3wHEZjl5IM0nGzJZ9sfsD5r0RzCgIhWPdPbf6WTOlufhPVi3Fus84VJNiOdCAwP6i89/AHrrVdj0n0BBCItxWizxICWysy6z7tDfMSrxnnVamdCGifWxPngbsjZbYjMkk9c9WQeKgMSTBt0WythI5LGuRToSGAVTDxKsS3nBrXS85ootWV3O4g1ajisqIidO+lC88/FpeWsqOzc/v8rLBnJho6/xPczTrd/yD49/M/+v59oP8C6Ow6afz6w53128sSJuC+5bcWs1MHE9LG4eBBrUqurE9+KML0Swg/QYJmZ5DfTk7jqLOKKNHo4+uxyNmpsOSudPL2flE6MncSOnV0Z2TGc9XELKzq8p5/sPvmolXW1H2QH90a+75qqLM7hGZYz/0iIj3jEOP6l2hGYJHa2/QC2mdlI1BEraZVZJiIeOOVN8sovqGCjx01mJeOnmET18cW12l+js3c1msR2Yt/77JhBbB/uao7ia21fWUMc5mXIdjwbjGOu05LAJJJYvnxuZOQ3i5q4FlkvZWbixIunmcpqeOUViSArEVLraXmLdbTtUE1oHRaRrYxybYTMCYs1H4wrD0wyieWbmnZEszFBhGDX8mmdla2auKCwxk6exkqmXZsqwuIhtOPNr7HDbc2qzM5IiSxsfWScjnzuRFaFJJbvE2jUidAsssKr1vpZrYt8joO4oLLGXPI5dmr67Ej9VroC/rRhLZtZx843Wdv2LYkkMgnpFLHVRQpl4kdEYvkKDUSGQECralKzAhcV7EzxsZ9fQLvpxdY9WimTuEBaZ8+4gR279GZ2smQM2eQuGHb8CDvrnRdVkBmIbKnKpFHjuQlTkhNbJFK4lCgGEnMitXY2sB2KDVeTLq+TZIX1KmNnuiSIOrW1qAXLO78GJsk5j2jhpM/fIkVpjf3g7+bPw+d/JlPKbMiOTWzPX15gRz8+KOtj4exfpKL+UiD3M9kEpgmJ6YAZmuTxVFjEJSUd4sLPXsdGfeYmIZ8WfEKf7n+XnTxywIzcdR4+6Lto7ZyvocUj2chxFabj/+j51alUeCDwY39ex97/m7RZta9aRNYk8TlqDPEMfSmuovbAxdwZJ7HYa8By/FxLwn4W1Nb5M2pZ97X3chGIrS46W3dI9/vgWMqnTGNnTbuKdV50TaoIDSbmiC3rZKqyVZZp2S7heQpDYLHlgoXqRpFREovdcS/LXEQUcfysOezIzDu5/Tv7Nz8facLn5MtmstKqK7mOMUkYs+Vp9vHWl2WkZUgxKzNJYBGRGHYZmElYtHE3yMPDUhNXxMVSXXhQwnQQMJ3y5dfP5zITobZOv/EUa9nycqxlODA5J112JRt+4zdSFf2E+X3wlbUyiAx1yfVBn83MEphCEht0Q3L6UdmvKEeuIxJUG5ffy1Jd68OQuAhxQXEVbFwm028jDSLnkTEi67DWzPoAz1d2CUwBiXGRhZWjZZOZ6g4KsaRMyFBdogt+1B+Xs12vb9Sh8JmILCI1lqkoZAQk1mGZaq2C31/HznQMlanOYpnCYhH0+qDnAod4xQ13cPuPsHhanl0uM+wfGZGNvu37qTItx27fyHa/8Isw96LNIrFGzmctG3lgEZJYKFlqLf56CWS2zVKDkfq9jONHhHFFkN+Fv+jia25lnV9czP07IzY8yt5748VEL/qpV9/MTs6+P1WRSwlq2HcdScjETw+BKTAn4YQBeZjZ+CxARn4OmeEl4kOK3O9lmYwNQU1i5HH13vog9yKGk/7Q0w/F0UpGCaA6p9zydXb4sltTQ2IS/JH4xTqPJO+wsyKTUUoUI4m5KaP+cVe85qZ1s+o5CSJSv1cYkxELt/K2xUL+ICRYNj/+gPa+riBA+sWQ2x9NlRoLaeK7bsZhe+MnophbUxLLt/kbGWerHks24/jcCp4jzfcKU8M47cbbhcxFc8vc8jR7+3erWZoBU3ravMWpUmO2Wdn8p2eC/vqgTTmkAz/WvvjKx6rFmOzK1a/fpclfpPleQXdAJKJO+PJ3hWsMs0BeuUijbwzqed/vfxzU9O8PSknoyBpLgCsyAouZxHLJrMEvP8Y6ThBZfRROyTD+riCqK4vkFZbsU6zGzOCUtXmHWZuxltVFNthWwiRgWWZmg0VmrXEeiEVejaLS3XRS3/VIoIWYVfJKu0kJNbb7qSVBfGPbrJ9hRufF2tQg0sncxqIFedytyX1fw/oKYSMnMstZD/IS8neJRhiJvNxNyq65D6eLnI8fYaeeeVhFU0UvdBjrpyzO846UwAKQ2D2sL3XC7ttlv2Qmqb5qEVljROcvTF5QDlVzFwQuaE5ztDGMSXnOnY+lrqtsxBtV7I0NIicwARJrMy5OhYf5VZvzkjE9GkRWr1KRBYk0hjEZ7Z157w/rYiMvHP/4qpmsaMKFrKB4FDu5bxc7+N4WLfLOyKQMjdg7EsdCYNZi9gvdcsvTnEhirfUzTNcKmJaLZEcggwQyZOQynX7i3qhGgw0ih6mzv8I6rr3P8f91KltKq0l54leLVd/7KXH7kuMkMB4ndqAWIVayal0IMpPagzwIeclYVCHzhcIpm/tW+KpGLLLOX3xTCzWGesqiry1PXUfY4eu+p6qjiBYt1WMjsBwSa/UhGZBJk/VqZ2fKioAmzhyvRQHNzG0WgTaFOEdh8rr83+4P3cAPZsS2Vd+K5b5Wz3+I2yyL8ziDEm/SoMgvpsVAm1gJzFrggSJyeXjVIkKztMilXALmJcgkSBQ0UGG5KHnJXEDdP7kjFmUDJVO44MlBJIWe8N0dB83pRvnk7KcSYEqjb74N1T60S+fUu5q+SQW6WzSvWy7LFwpRUaHDTNTYCSyHXF6R+JF2WdHKfDKzSoiWBiAyITUWJ3mVvvY4e+f5eDbHfPXoFETIT8KFP6zpiQcdydCtTQ6K0I8++wNlPp4wKSsZUGKxZt/nolCHg7BSGO6R+JGTLYLaik6TuSPV4HS0Lv4U1uew57aMQIpWmxup5IWQ/gX/sV4KeYEw3tv0m/geqLKJA/4++oNtg3b9jrYdA/6OaUROBAIl55bmgH/H/+N9KgBFCP+cPRIu6cBzgXkGkrBUl/Mq1OVALHt6jYKPxhP+ikVkNQ5ENsMyQbnEjfFagTQQy7/mZhILkdeor/+XtJ0ebVd0z/fCKLVc5J87rknPvB/xPTfG+6DUVABmKvLnoFySDiS5SjK7X4078qglgVmkUs/OlDeoIDIospW55AOT0Kqm/5JlevLgbkuNVTiQV2Nc5AWzKu4e9r3dRwf8He19cJ65pnLZvwwU2zAhB5iON93neG54H37mA2amKmAzgNkFPx1UTBKBY5eYob9Up3MbquH1RtSwiambQLTQ+o6KPPJcbw02WMr4OkPA7mmCeQoStMiskfe4ZZOXeQ5/+nnsN+9Y858Zy4tAFn/nt6x6+0aD3DrZycpZ7HCeWXi8+bUBBJff0yzfp5fvZ4M5ieup0rGPjaH8o7bEFYQrSKNo1+n8CnW74JY8rYvpu9uN1yLLrORRY6WWqsPvcE8MUkFeOqgve6E7+Y2QVgHSyfdpQdVgZJsNTOvO///8gMTODU8MVmHj1A+oAkEi5QP5dboD101RDlijm/uECOwMkUDJrFL4FSt9vh8KsEbgGFYwzjwzFeQFYHajLkAZC6+5hWxxL59dUdcRR7NukCmR51dTCSQHI01FVwc/NjMEIBRtaKU6kZiWBGaRCFSNKn/Yeo7vt9UYfGMdMr5UFXkBuSombqA86NCq+YN8W/mLzKnMqae7a6BJapmHuXBy2h/7qDXSc8xVYzr5xpDv9Y8V96jOAaz2EwFRQYs8MFf7vc+vJNsf5lok7nEcUGMNLETRuEry0rlVDsgGyat2egWc/PCT7dn+pqvyqnrw1wNMTZAd/HtIhC0uLR+Un8VbsI5j6Tx8UHr9pej4OlWqq+uFFVG304ltoG0iCMwij8DjxVywylJWoscRuHuq6hKVuAq2VQG5XbxpFIBXzacTudiEKNvEimPQLsi7+LUn4xxE/KUgk8AzQ2AWecAWkZWxGGqGnWhTRtXkhcW4c9lXWdrAWw/qpT79VC9M3B1rlkhf+Pje8bPmKFVkpgn+xlOm6yDmvL9AQ6izRmDSTEkZI6BECFWksDkI4iwbUg2/mkSvc8fGgeoGP5NdpfmNY6iceQMrvuJfpWxgIK1hLZvZx1tfVq64cewCxBhbZ4qhSXiQwe5WT/2wpmRom8EyabnVYEHxaKXX5qOmRpZWgJwmvvtX04dWML7KJAFE/k61/JXt+csLbK+HL8vsRZZDXnBuH9q83oxWIpHWJhSopImKCAEEYE47N14ghAn/53KzKH145RXsxNhJvt1goRBPt3/ITux7nx1ua2Z7IyzOR6NH+Co5zexqJIgHcc1kQoHlkEeY+XVAKKej5czfKrqTqTIhzRyqh2+K7X7Y0UAd/W+5gYD8YnHck4se+GX//8cdBMmNqh7Y837spWC5ylfQvxq5PywRCiwHYPgwXSsCy5UcJ77wLoz5fSoikKP+8XrkNwAO9rOumjeAkC9gZ1rm6JBMC+Sqm56WtwbdE5hizHpPfgF61NBpA8D9zTXb0eRx9Kr5vJFb1AhH2mZH2zwwF1MSBLQm5O8HxdKg6g85OUjYlG6i7NsV2bVHNK964c/M6KCTmsS/4f/wnvy8LSfgPeg6q6qbxICHvGRwkmvcpKUjnKK/2HQxk4ET8FFHqsAKE3idocKCJJYGToq12vEsDHPQ2GVll6Dkt6VRSV7nLFzLZQbjPVCbbiQG8+SC/3zTrI9Ey2wsGBUkllv03TVz3oDjwfflpjp8uv9dV5JFNLTy0ZfMY0ZARlXnCx3JK/ee4jpw4jqellOZJTBLngbJAg7UFlrEdIRvxQvIVfLKThcFkjKjQOVti4XMX7wXI8ucrodT0AELB2pMJgr+9tyA4zELyg11aKvIXCAg4LSg8Ttw8tvnjmgyepAJLObEk5cNXAeBjWZpfqcWIrCBJAZzrk3w14L2tcd3+VYKY2dG2N7PfELekazSkygm+qCds1NiJhQO0hjgAHc6H/igLr7mVkdz2qklDtRYzYJlptqTAWwW+bWKUBL5KhIFz/nXEefstaAFF7PWABnzJg2jAoLz/pSyAP7izBBYDrEoJTBe0xFKw55o46Y8+tXAiRNmc7mwiKqQuGzGFwb9G0gLybNIc0D0DmU8TsfT+9kvOV/Xfe84/juIsvTB581FxeNHs8kGxOdk2qEZIdInHO+ZR7eGEbc8cOY6G79fuPb+Qf3Aht/4jUQTF55RmMQiybaC/rBITMlEpVE4EAzsEd6tcEyA8WytPOorP1kVD/22tY95/k7YwRFufeSlm4+PvjTAfHTL/MfODPLJR8eyOYMUTn5PfNdFZhAGIq1OwQrkUqEVtX1sXtcDZFg+dSYbNuFi8+/dLU2uGey5Q0ny7yM+B2aljbgGp4RF2KnkAuP6lA//GMqSDagwnrSKjgDkxWU6wpTIz7TH36de3dSXxOgC9K2/aPrswA8REhyjQL7va8iOTa7m7BRDheWbaKPGlgc2dfHd5rW9zP+9UG/wozldc5AML9GgWLzH/v484sRnXDDgGRmRuAVjbx7HQnwGfr+cbzKUbUoq6++XZBPSTovgSTwSMh8tB6Sv/IXqgF/AcfHNvt/TX4DdH1N1guLTw/+M5ZoXlozS9nmAHy2sbwodL86c68hBZldSAWWJAAaP8uUBOtNyYm7uUB0iMGcVJpXAWF+U07fu0is6h3/H/3sBqRW6D4zId7gfu/Rmx4WMBeKUZoHM8sEkqK75IBzSMM+Dkg3uie3rQvqF7VvD51XNXTDAvE1CBxBsovApwiyWWQ2Cz4Ka40QDEVg4FcbtqLB2C9+WOW7RuXyzxk8RoD2yzsMi8k1GEDNKo3LVJRY5ghj5gF/KsXvq+EuUHjN8iygVwrX3IjK3/xu2aXX/uWLhIwfsvB+8OnDe5abVWq8Ls4TNIBjk76nqimGaknzBlsmqHPqJduLnkY6XL4y7hQ5PvSVvpwNbwaBDpld9G3w3MH9EIOBIDb2DOznnAUQee0rKXP14iN45NdgDIUQJECmSVU8f/6SPQMeeZxaH49jdWhF5tfPRuYEk7tekz99iqscohvIKBJOUOPSHshQA5GQQD1TYdWFMSGsgrW+5UH6nAy/Y+VBeZAPHc7WkliuyAQf8pNced4yYeh2vGcFzIC8o19MRn4OplF3UMtSjkykIgkKXinNm1fUHaex6z7c1qffMv65IecGxdkR8bd2CJ3mAS2YRkzyWLRUKzE+F8fQAszLuQXSekcfcMDu3nOdoeSz6uVEpMBtwAPMSLBY6crCczhc5W1F2LOVRyEltCAnSKq260vRLRqG2wjzfOZgis/lhGpz4/SqMOfvCeLfLRYwjbQItg0WBh8spKz0XUAEyy4xkg3dCNZSXG3mBpHUiL1shyygNggpBzhw+C7433kRcEeD6wa+FvEOY4afnrx5Q6hQX8P2wSjhBCkxQha2xJn77qa9W5hN5FO3VrlKFRa3Aco8x16yyz+2sd1707BSqurV2WITxa3k9F3ZTQqS9fPJRK/u0u88P5zRcJLea4KxxFWxI8ci+5Nsx5ydimK5T0rJqFTaUpQguvjAe/9ciP/LCAkT5SE+IXQrthb18BVj8lxsLKc7pNn7AMfaR1GNCv4cUBJ0XIa55dfEo1rxuOXdDQTwTpj/Uo6IiX3EW5jiEvB64rgSuP6QNcTr0ocLqyYR0JyPGS2CW+vIN8cIEDJo133+xr77L9z37Nz/P0gioG5TeoKbQrT4xbkBVIroMM80rCdkkLsNkRKpGmHKwtAFkzdlu6G5Z3SpSZULmkBISUVGEjRKiMp/3YjdY4rfT8qZN+JKYS2pBLngc3Ukf5oFrOumyK01VG3ZjUAUEI3r37+yvekACLnLYdPPj6QSBtApf107mTMg8iQqGbwyg2BzVV6ckRymiRsyHwA6+spYV+iwS1cmgqgEzzewEYbygZlB6FbczepAig8mbAN+TbioM0VGOAbtQYUvD+sLSaEKaTQ+NV53xWumjvuoZh++r+9p7pfpa/Mpc4GOKql2ODoBf8NCq+Zk65zQjtx0Rh9AIZ9Fk/Fr7XkCoL9nKAKaTH5Aw6QW0kkkTEL3atupb2teGEvwBlwBUGAfqLB80EZgojAuHFh+To1RfNs6adpXve2BeedVI6mZuyQKc/URiyceYWbfxvM3OzicCCwAu35cKokC0i6dbwogt3iosrQMmiMSSD4GIZD0RmLj6qmAcnVxVqC8bmNLsB7+p25gyrRtAzHbGuP3C30Xb24DEdK5MIPiDs2plsmUNBcLQjF5bX/VlZlcrNNO4opF7W1m1Q5dTGxhTz/yjPZEgf+BtZ94ueR7rKzMSGFfPWp5dzs5ZWJ1aczkLKgz5dBzZ+ViPgeZJZtWE9JWtqoc2nKycxfW+7rf+4H6MlVfEv8teUOE58DbfdMb7kOfGUyuIB1/3vlsEb1TccAfP264LmtiaOQLjSZ2AyaM6uRKfzzOiav9Od4UVdyTSnp0oWiKEnRnDb3naPyPFwmkMGyEZcOvgG0RUEIH1wdfeRrFyFBgzgU+FuOVHwbSSNUcxCHkFLWy3j513KnfPn35OTJBQ2DXARGBy1BcYw7NdNAghf8qQKpg+LJ6HYId7s9nyKdFHImH+hSGvXGAoip85uWf7m1q33Sb4kAxHDTAL6MzPmgLzvUDjq2ZGdjC8PqyD77mbkSMrL4/8ImKmoMwdeuJN3gXRKDtCux5CQs3Isyt5UyqIwMLKVM7dQgp4fViIRropEN5ggEzTkdc/CNOXRznxDD/5pOVtYoIE4+wZXGYkEZiP+VjtZxpF2RlBJD3ATYHwBgOkXUef6Cwc7miZs/ffrzRLg1oevslsdOfXQmfUZ27yJvHdzcQCSVZhfM78UlEzMksKzN98nDUn8oPizab3UiBR+cH8CN6ewJSf52XWOa59zDO7HirM6wEPOt2boAewWfPUAIuqsCwRmK/5iF0iavBm0x/9qM2dODhqK2Vg7GRvosSkca9upsiu90qJ8KtOoMz8ZIPzOSUCC2I+ono+joxv3kiklx+s86JrIjnWIR5kyzupOn9QbpBrQUgmOGuAhczIrJQS+V4QlPboHqgf/cE2x26gIF7OJnJKj40H9nBZQoxKyFDBw1o293ea7WjbYVoCuRuIqmEiMCM5SsnwkHOVFmWFwGp1NB/NGzD2PO739rS85Tqglae2kpBdICKMsjRUdux18yc6PD9QTDDtZc2fNM1IfwKD4OBqs5MVAvNMXoVzOq6C4cKyidzvxVguTwJWPO7+VLe7erKd8H4TfcydPcD5EYIT16EXf8q27QoWxcX9NJU9XsbzhXSXMHMMON0dSGqt4Gk3nXofmDUr0hPjamoTcS5d7e6RONuMVOrDaPNeBH7De7FReFU5HNnn/bzSMA1+wCc5YsOjZirLh7vkpaDA/MMkc6TKBKlRFXhOufxgWVBgvhdiSOXnEnEicORf4PH/qs1IfH+V8dC67b6dX1zMphoqzWn2JcgLBdwnPZTCXo9UCTvXzfbfnNj3PjtmKLYDe953VX1IUSkuLWcl46eY9/hwRgZ04Bodevoh836pgtkxePubfQNvjPsuglEV03meU+xWK4nAfPxfZu1jgh7ssR79wcxkwQ1PcA9mDYKCvz3HmMcD2zX3YVYz7Vp2vPk1k2DgHDYDJDPvdCUv89h9ZgAAmCu5V2BRnlEe8Lk0mPcapWKottB1lJuM56P58QeUPgO55iWmw09s28GKvrac2w1zavpsxvxHAnLJ7SwQmHf2/ZRpgadtx4GC40c95TlnlCcwdr2+kV1w7b2eD6tp6hkv+CdOwzTkUAw7fY7ZTGQNmcyKzzgKdWi8YMagb3uazNIoySt/oxi9aj6bctcjXGLArh7xSU5GOkVNT/dxz8HUqfaB8fi/4iiGDoNP97/rLc99SnJk7LqnnnlY6mciATZqwDGNAazw5aSh0wXOYd/vfxw5eeVuDiBP3tF4nNUjvus37U583wsQdTF0WPjlUdltfFUvfkwGl7LJGAQi08kcxJez94d1vrWa2hPYptVKfV7cJiUniXEKByIwr//EQk+aL+RE+wHf90z6/C3Kj+Od5xtCkxjIS6W5K7LwUKuJ40kiUGLlFDiJ61rufmqJr6rlFA41WSewGgkyVit0d/j7gbpmzovkWEBihWvvFzbB4PM6/cS9WpBXvhpDoCBpJuXBV9aKK7aiIjOn69I59eaMgsv/7X6u7ri85uSJX3lHJiEcOMqKJvv1yk+tE986cc/e9wivx+3AN7PrZe+CJWPMhzEKgoA5Oex/68xwerePcx/EhSgmAgFx+Wp8yQBm2C++2ZfykYBpSFBfTYImOEbd4V5h6taA53/mnazmM41sx5oloe8P3ALT/rjcM8UCGf4c5W8QIa2ZIzAe+Tl0/CWpPXmMOGMRKRw7nM4QUr94GiudPJ0Vlow0ry8IGhn8SILdG7OPJo0k1vn3l4RU17T7VnhGCuFDxXtkRDPxTFRPvz7sWECs4/VEYC43K254lec4AYmbPNWTeGhAJlE7yPF9UX4nEmRRSQGydLqfUCiI3GJIsIiTG+8dvXEZY5J6/8fpUuAlr9xnZ/rdj5hR2rBAGVPhgiedrSSzpfozodZxmgms1u/B1wHHBOv/RHZFtPGNM8KnErk1eR1+m5TxKr72PrOKABOOeE1rvO/S8Y+zjmvv0/Y68PaTmzr7K0IJ27huMjZA/H719o2OJWScLdU9CSzNTnxPhho9brIeJsBhdZ1Gkf0e19g1lYrLHqQrGkHG+0UG6wIIVPDmNsUBs3zMB3gGgpAwZx97X+x+4RfOm7FhnvM48rNKYJ4nXlR2rhYHqbpVchQpFVGqriCDdJ3UBe9gXdsM0hXYpPzIeMotXw9mHUhqMYVn3K2b7rmTLvT9fa+E9FQSGEoQfN/DOdJMJaLY2ZFSwTkZWXvy6pHojxIZrAszyKuff9wAGbt1eEB6RNA5pzIDGEc2P+usICdP5/n1MlcTOqXqq8zvDSfGTor9IHv371T+HXgIkeJgRgmJvAYBnzv5xP2+4fzWl3/LSg21oyNMopm/mlUbG6I9BBlRYBRNH9EkURvX16mTCY6TA66RyLSakLU8/pC4gZYwUQA5P0lVYTCPML1bJYbc/qivrxBmkO7lRjCtkXeFF3xeYZ9x2Qm9TvMQOFOZKjJlQvIsCi0euLbmSI4dO3TlzBsSea8wtVt1LpZ5fW7z72nVvvV/MrVOeOcccJuR7/41qCWUOQLzVGDDi0fEfoDY3YIU3wY99ignjss0HaPK1bPTBvzMoDR0ruAFerrJBHyJ+dePUyW6EthQlkGcNa6CdcV8DKP+8Xqk34cHJaryImn36ap57LDgpoDrenLfrj6yr7xCiADLr59vLDJvcxUT0o9o6gsLda1Rn/rGU2ZeosrcQafJWhy9wSZnjcA8o5BDOJP/VKK7pSny70TiJ0sIgZn98znTJUBcaCczuCPDM+bigHnIQ2R2KyKvxWROSE8RgdnXbmdE3SycJmuNGlseOJ0orSakZxE3Z+RDKTDeKtCJ8YWdPVVYIghs6kzuBdj5i2+6tpPBwkBJDG8aBFpOe+Hg7vRUNvhdOxUIOnnKLRcsk078uIu4kf+lOoHVU4UlALx5eryN/N7+3WquKTrFld4phHHdNxVAy5uomyA6TdYKsymnjsB42kjHbj6+9Yfgu6bHXMU0qTCeOjkoCBH1gDpI3/dMuNT3PW5Z5YkyUV57PJY6WdmEmUkF1lsyOpHmI1BQHP7Yk6DCeFIn4FAXwZ7tb3IRfBbw3qbfOP47fIBTr77Z7BnmF5WNGI7SOJNRyDjHqGH33hvCDOGs4PddpHhAk5ydD3x6+J9ipKhpE8XIXRjbN7K9DtfCnt3ZZW0eUDc1u+Q0OPR3F/g+i2WkwDRAkPa//SZTUZG0pM4kZ+fHDRVddCNVt1aaST7sBo4wze2qA0Rm/Saux4k0EliNrgcG9RXG78BTuS9ioun8YPI43EUL8nmqGHgK7IeOPS/RC6Sjbcegf4O5aJMXIpMYcmJHbrHZyYJsszSNBFam64GFUV9AmGhN0lTY8H3v+L5HdIQcOrf64sgH/oumbGJq1eWoj3ez9gP7zT9HVatLBJYAhFVfpgkZMgKZJBXGm+jL2+sK6ounqV8cCcaRq9txg5UoWpVDfcE/jNbTcOR3ze0bYCwaLPFUr5KTyDNHYHFFVsKqLxNjzpdPFJqqMN5ILXpdoeeVH3nBvyPre2UEUuJE0YTBrgg46QswA4D1Bbls8gKpoZWQLIwcVxH0V2uJwGICHKIycm5URE91VWFeXTwHWX0z72RVD/7azG/LJWMQF+YeoosrT/AD98kvUVVmICUuuHVaRZ0s5nXiOuDaI1cMU8tlJu86+Q/DpDUNJXpRD7ee4CJQ2QLIVGEazmqEai3kLMY287fm/YidlzfTt0Pg+w5tXu/7HpmBFOUbp9XgMN9pXzx5eh8RO9xv1VOlnPyHYTZmIjDFMHcxCTsYhpCoGsILRVE1d4FZbqMTvCbayAYibm9zLNwxl3xOiBTjUvzYNN2euzgnVbmNvtsblBCJYtTBLHVxyXgWBaaIq4SuE4ya1y1X3oMLKRs7NzzB9V60adb5eStce7+ZAqFjzaYKKyJzBIZoS2QP1KbV0syyKArQK264Q7v7heuHvCSVi/7Q0w9x3ScsQF1Ljez8rbbtW7Rde2MnhwqgNRGBsejKSbCry2xTEkVnUl1VGAqAh6/7nnQlZi963gLj8bPmaPtce50HIu8oHcMLgY647rFfpw8ftDtu7CnkqHYdDqLrhRVaS28vFaabLwxAhKz8o7b+chcZGwyUl0h3hJOVs7R84EdseJTtdTgP5HKdnH2/eb06c/4dzfIqdzWylmeXR2pqqvBlplGBxZ6JCKekTCkfUnqnQoXZSgxh/bAzGvH7/1hxj3BrF552PFHDTekjNw65XG5kD0V/zsK1kW2ObnMricB0XGgyklblSW9h6DzNGy4AKETkK4kSGd7fsWyO+ftBXAlQgbr1AnMiVZAXT99+kBsUbRQkVlp1pZLPzeZQD4cBm7LAG44X8m9cdE205i+meW/6jdbtZ+x8pWEbnmCTLruSjay83MwxQpY8FibucdHhPex0+4dmH3v0Atsr4XxEctOiQH6PM/i78skLpNv595dYd8dBs54WeX+2MsNPjK47+IS62ZvIOXNLngU+3f8uEZgI8GCrIjCZZRcAzLmoM7+TNM0bJGtOWopoWAlIEz2yPtaAxJComk/K58yqGzDJCRtqU45PE8c/emujaT7azxXOBSpMVXtpbDA9Hs/w6eOf8HxMY1ZMyNh8YHhYZDtFy6fEU7sps4UKuQgUwaFzRq6jHCrUKSCDZ9Sue+x/zqbOVHaYoz5zk7LPTh2B9XQf941CwqxIgvoCYBrFomyMHTMpE4ziMF918IXlNybML8qHpeG6QXVEE32EsvNTq0EnFaVVgflCtBVxXOrLfEhjDN1jsCxBcxWWZ07rBp7cuU+7uUzITCWyRj69VYX6ijvzG0W2mg120EqF8XSNVemu2PX6xkH/npvsC+Xj1iopvznmKT4SEQL8tzzR0J7ursCWVSYVmFNL3TDgacMS6HMnx08eZ8+4gdjKbVHFlBdmRrpdUkHymw+iOWE+icE10PnFxQP+LcykLDecP6OWT836Bw/asmZCRurI52nDEgRR5385ATsoDf9wBlIYVBeau5GXlyWQe0xQ0Rf8x3ozNwylRNULf8Z65v1ooBp77XHpGzCeGZ5AEOf1c2W4tKZRtPvJ/wskfZHZyUBRe5Ko87/cUDnzhkjHzycFUEBQPDxmUmAFvKvRDDrBbwsz722f+wAiKkeEMYekEJBxO0akYjRL6piSC6ThdHKk/4z+YFuo9ZxWAvNVYGB+GflVp994SskJwP+lS+fP4iv+lTEiMEfs3/w8K5ZMYHbiadDkW+TFXci+N0hpOZLX4w9Id/7D95VvorquH76MgCZSYA7MLyMZsWXLy0pOAP6vLk0uJswQPJQ69piKG/DfVBtEIKPdN4gL0c0mCYoeJDZ69xyzOD9ffcFqgP9um6LkX/M7Od/LmRGQLROyp/t44/DiEu8Lh/KFkATmNuFYBjB4oUujazq+aiY7SirMEd1v/YGxEAQGawC945okX19sOKa/zHjZ0eTOwweldAh2g1Mpkxc4A2quBJbmKGSbp//iyIHQX3Cs+c/q/CuatW4xzUiCsxkZIoJn9yRT7WO0a0dVq+jy6+cLvR+EGsaETDOBtXqST4jsXxv5hbQyfQi6df6EiUTRSHelwzPR2428VNUgRg2kZ4i6ZTgItcOruibNBNbotyOFMh8x8UWR+ThmQoWWFxRFuQQPM1IQaSIvbG69t4p1tOAsx/IMyKW5G4XvkzE2hPP1VMtflR04hn92anhBzbrMv73KCM5mZOlc/ve7dVF1UuPwP8Inao8ky20RpEv50NTZX2EdglFzzjY6jVklMN9Uit79OwM7Xztbdyg78OGVV2h5QQvGVxFTeZhCUzg3RCgPP4c9iMuOICKYMyigY/w7ZmAisRUTleIkMjjuO669T/j3ju/fHXodp9aENOxmXwLDLhYUR/apk/66jq5XMRk8TeBV5X6F4Gi/jH5dPNE8vAeZ9nHWrJ5z87cD/d7B3VxunGwSmAVPe+foR23B7P3jR5RFc+JoYCiCKAeMJA1H3vUnMLgtvPyvIKLT81cLPQN4b+GCJ5X1nfcCypOCbGycawgO/NYsE5gne8OBGqSWbdQ/Xld2wLo68PvN2+IRxFQu4AkMHfvzOvdFXVTEir62PPD3D7n90UgHsohk3AdcQ75WVNoJrFEFGeU3kpMJOPB1Rn4bFsJA+EXWvMymqrkLHJUXMucxF7P7J3ewf37/OnP6ttP34Hcrb1sc2bmG+a7uliYp6zfzBMZ5IQfq2rbsOfAJfOhpecv1/0BEbmYT1JeTzwsmJ0bAoTQIFgOc9RjZ1/TEg2YXiXwgDysKFYaZk2FK8Q63NUtZv6kmMCsBbpuECzkAnNnDgdBbMppYIMHw2ty8Wjw75djBveFVbP3O8w2OSoy3D1cY0xEDc4MCRM6T/4aSwKwrMF8Wx4UU6ayp0oFvEipF+hKNA3ved/0/r7ynorJzHd0bfukR6FoRtYqH6Rgm0DSsZTPP27gSDjNPYAIXtG/34etflFp8IqEEK80A4biVFXmNDxs24eLBn8Xha41qOIcs09F8hvjSlxp53kQEBtm/k7+mUdVEo1yfh874VEHv9LTBTJAW/Z3uwbUXhSUjfX9vaPHIyM4LfrowpqMNzhpirjbHqScwHj8YnKK86RQqJhrlouD4Uc2vZxcjBHtGho49T+h3Tk2f7ftdpVVXRrbJTpu3OHSOIlpQcVQNdPAkomdFgQENfm/gTadQbUKFGbMeBdJSfKwSbo58u5aR93fQkQQmmxuQVOwUuQxTYeIGJMnmDs0NCs6oP/eQiawQmK8ZydvbS7UJJaNPWVbNW13gFqX2KhFzG9PWNfdhRxJDxv6or//XYDPPsCRkt3mC6YgkWRng7J3WyPt5Q7PwQEGODi8p+ZD1soledvl584I/nLKAtI5iTa9jEN9OFoEodanT5mSYX1BNbiq24G/PMeaQ2Q4Sq5x9f38A6cTYSaY6O+nwGcWvPSm9sDtIpwm3DZCzGywpsMGrj/0/T+Vj3HRU9vM8nKpNtDgHpnqazwpMk7TCLSN/XE2t6+80/+kZV5UL8kP0Dy+3Zpd4bpyG3YYBCDdIpwlHK8ejjCoHG7waGGaXwDj8YCLRSJUQSeuIEqo60GYJXTPneXa23ff7Hweqz8XvHHr6Ienqa+JN90X9/AgNWc0MgZlRjQLmGZ7xi0ZGNcQUo7p0A2f0iGA/by4lRWa94swbPBU4OrWKqHCoNhXdXeFnkzG5S/D5IQILakYCI7a4y9yoklh1NCNVDjDJGvwCQbj/qH8c9cflvhsq3oNyIxXR4dG3fV/aZ7Vv/R/p5iMwNGPPDszIhZ4y9y8vsFIXmx/OU4xoRzi8t/uomSmNackYEPKh5OncmNvHfAaTRrroTnSxS+fUs6HjL3FVHCfaD5g91tKWagGT79xJFw74N5QMeSkKpEUUhjCl8NnwiQ17faNZJ1kyfkr/tce1RjpPy/Ytys4ZAzpkDZYB0XIe63rRzy7o7e3NFIMNLy5BIopny9OaBcsCSWdIeQx3QKhYhrO/6sFfazediPeBPeudF9nHW1+WTuxRkhZMPYyTc6tPte83hhvnkxnMLzQZdDKltq19TJvzhJN+RFn5gDZOIMcRtzwg7dkzVaJBxj7oMF4VogosiwS2yPixwm/36Qmpfuwpy2EWsIzjiBtY5Ide/GmiiAx5VyiZ4c06B2EjfSF3kboRGPp6vR/jYBQQMxTdWdOukpKYyoOOZXN4NvQ1BnnVkwLzJ7Ay44evN16W+gGRffjS44HNquqFP0tFhwpch5ZnlytPQwm7uKff/UhgxzXIGlFE+15f8J9vBl3MSs7t4mtuZd3X3htpy3IBxTmDt3woF9ly4rP+2sg1vsyOpEIJwGIo/s5vTf9RkMGwWBBpAK4DBlVAVeoImFIYjhGqSZ+x0SA73q03PQguDvLCNce5of1z1PMWDm3mcmu1BSGvTBKYhQa/NyAhUGbaBJIBp923QngoBnZz+BDSACwemMQIhOg05Rv3BMQjY3HjMzCUw4moVc4SdVNd1fMfMq95HINi/AaY5GBp0O/IJIGZnR59csLglIUjWiawQ0ONiaoQ+FY4pxgnAihABpnrQGIyyWvAM+bgu1Q5S9TpvC564JeR+bmcwJl5D+f9+qDfkVUFhpww3ySX1pd/q4ZALRUigh1rlqSqmBpkjgUW55g2VeTl6g7437cjPa84I9jIY+QMVqwXjTzmInNOfBumM7+AfWAQmWdHOBANFAPI4+SOVwa0PcGEnmHTrw/sZIeDs3ndcu4M96gXnNNDefqNp8y8t1ygqd6oiulm/yrRRQMzXUUWOY95BRUocu9w/uhrH8RPht/dueyrqSNl1/XFH22d4jf7kQjMncRgey/xeg8GGIwaW+5py/eHpq+aJ0xmIEavwQ26PKC8I+xhHg+/8RtCRBYHidkbE+9Gs++VXw84PtHzhAsAk4R0JC9c/9wqE7T9CfN8CZA1Mu/rwpxz1gmMK6VC9CFCAazILi1KYiDVKXc9Ell6Bcjr7d+tFvqdILlUUZEYooRwtIdVEti4MMuRhwgxAg1ThHQhL1xvlM2h8sQpMopnbHzVTFZ49V3CqlpAfV3PM3mICMybxPBU3S37c0UXsCiJhc1Z4sWIDY+y994IFswwzbR5i7kdydi5UQOosmgcxwTfG8+i5F2IPJUbKhWYKHmJui6gNntvfZDr8wXU16sGeYV+eLPrxD+DpSo+FIv+0Kr53I53qCmRyBwePiwIlSkWYcjLPkYkMeJzeABSUR2dRDKnTPICEGDxK74HwanIgcO1mvDl73KRC1QXzgv3RGSTwHXY+8M6rki4WcMb4brLPIFZDsQ1Kj4b0hyqiqdRYhASA5BigZHzsiOUOOYw5JVP5jhGnry6INeAFzCLkInOY+6JlPuADI4++wPf90HFyDwvkUCEbaIHLWOyN0yvZxnPIOfnvxrWdCQCi0CF2Tce/iOnMfCyFjD8RttWfcvcXWW04cGDCIe9TNh9rnhJDOaxbFTccIevUsG5B/FVIcjjt1Hhu6EAZQH+NxHykuFfxLPsdp6oeY16vRGBKVZhNrAoQDAiJAbFIALsfvAhwWQLQ2QoX1Lhh8ICginCoxZhconmynkBxdU8zvYwpVsgfT+CRjmPjNw33iiqiuCIE4mZNb98WffS1BcR2EAsYgXsuMovAMGIkBhqB0UfdhAPTDY4UvFdohn88KmpjASafa4Ms5pHiWGByiKxc27+NpfpGObccW7oSuFLpiHbNCNAxJsCcuqZh5XcT5B17kaEQv04rB0isDMqrJ31MuWV0yIkBpPDqziY57vgt0DUiQcgO46+TVJIjNecxEJFPV8Y39G0G2/3NbVwLO9t+k3oc0MNLY9DH8cUBAgEYEoRD/CctSlqeoh7aKtVbHqcReobZKovIrDBWGmosE9Uf4koiSFnCd0sgpoaPGkMZtfMZ6MrGhfyiRnHH9SxD9Ox02FU2aDz37RaitmMz+CJxOGYRDcmPAO8/eFEAxFB7yGeY4FJSItkHwMR2GAV9u0ovkuExAB0s0CPMixIHmCxi2Sbw9SIutULFkDBxmXcJjVawogsepjfRV/zJ2UoT1kRV/ve8vj5MCyW53zsrhK8483gn1KZNJt/rpzEvypMyZAbMp/I6ii9S0qaDSKriuK7gnRdxQOKQnMnwrHLmkTKXFRnicu+BjzdbuEn4jW1kOKhYqKPU0dWJ7i1XA7ShBDEiYi0ZgjULpoILCiBFZfUGj9e0XUB5z6s6DF1+vgnbOjY81jB+Crh8qIoavR4IEI4ueePAnv0cMekHxSVo7d772e/xE3enP3aAwEmH69qgt8M80A/PfzPvmew8grhKgvRao4I8YBBXitVfDARmDuJQZLcHdX3xdH/XrcHXsTklQHV5C1StpS2e5mDbQZ51aj6cCIwdwJDu519hilZEiWJ8dacpfWBj4rIozr/KLqHaExewIyg7aJ5QE58F1gO/UidCXCIik5lTtsDbwc3VE5Bx2erStbNh0i0NdC9RGG2vuS1SiV5kQLjQFHJiDeNa/R/o/xO0S4OomYTio81feAHKJdz7nxMuvkVVwNFFUpMpf9OAtqMV40Kxz0RmJgpWWGYks1RmpI2EMnCePeIB4xqAxD51Nlf4XaE6648ZbVASsiszetlJ60SgQUnMd9huCqBXKGyGV8IrMi80i6SALOB4y1fD6VIdSJv3M8gk6/hWkCSbJyDcQVMx0VRfBERmMampNMObk9V7plwqecCwMM+ZMcm146bSSWySZ+/hbv3PsxFTJbSlbxBZKVVV7Jjl97salra59Cx801lZUFJNB2JwMRVWORRSR5CO3fSheafi0vLWXdH3yI9sOd97X1cYQGf0tjJ01jRhAtZYdnEAf/X0/KWmRuWkAXffz7Di0eYg2JOdX9iDk7p6e6K3FcnATNUO+6JwIKTGAYQPEdXgkBwxCMGeS2N8guJwMRJDBnFC+lKEAgDIKXHPRFYNKbkX6KqlSQQEgBltY5+oERWQVgJrjerbn5IICQIdXGQFxFYcBJrNUjsq3QlCATT79UY15cTgQUnsfW4eXQlCBnGhqid9vkgH1hIFJWM2GRcwy/QlUg2kMYwoqzcbMdzov2AmZKieaZ73NhmvGrjMh2JwCQhy059dI5AUm3nRdcMSMS0e1vpnnzplxireyJsjIDTvkZFh1UisHhIDPWS7xgkNjIL5ytSCgMy63phhVZEBuLCjEiR3mNxd63VDJEmqxKBRUNiNQUFBZuN61mc5vMU6TKaC7Pty7rlsVcIoPPrydn3B+oKoXnfrahwj0Fe2jA5EZhcEkt1pn7YjqlxEoDMThAZJjGtyIsITA2J1Rs/fknkpQ8ByO7FlVESW2OQV71uB0VpFJJh7VD3EHk5A0NHMOMxKiDQUPyd30ptJIhzqJq7gMiLFFiqlVgqaiZ5etSj39jHW182/3zWuApWePVdvg5+/M7bv1sd+7Hbiqr7rT+YHSBs8JzHiA2PSp0nSeRFBKYbiUGN3Z3U44fpBfXieY7rvjeowR78TVAofqrN6XejJC+/CKnfeSDN4tCq+cpSLHD9J3z5u6biQxT0o6bGqNvraE1eRGBEYq7gGQnmR0B+pqeq/vQ85CUyG6BmwTJX57+qQbJO1041YSaNvADygSmG9RCsSdpxY6hIGPICYCLCVHQD/FJQGCDLqE1ezIPkdcKD6NwmRZk+vRtvl07ATsSP64XW2kReRGBEYj4LyKv/PAiA1/TbueEJU6W4QaZDnJe8RH1vILqjz/7A9f87v7jYNPeiOAe0EifyIgKLi8S0j05iIWK4rhtARiIEgMW/+6klnnMRoTawcEMpRkMFqSAvG6iLxGAQN8hQkjwEjDkHRF5EYHGRWIPOJIYFiIXoNWACZCQK+GwKNi7zfA9IM6iKgb8IKkgVednAVCM3NRlWSfIGHd7b9BsiLyKw+EmsoKCgWzfyQn4WFqIbTj3zcGAHMkxOP38YBtmKqBi8t3r+Q77RTpkpG5joLVtJgoD9yMt04D/9kKrk2XuSSF5EYDGSWG9v7yxWwD5JCnmBBMIWZPv5wxA0wHHwkBjUmnnMPrMiZeebIWLqZUqCiHhJDOdQvfBnvgSseJq4duVBIqA0ihhhdbF4UUUrntxhuCCNY39e5+h455n+LbN0Bp0gzlm41jMzHt8HpeO0YEFuF19zq6/JqIK8cuGVWgEgbwvmntM1A3GNnzWHq7pBIXmhJU6tLl0liMCSS2Jlhjn53zKH5oK8htz+6CCSsIfdnj7eJ/yGTb/eU3X1my6Sc494axNBQJ+0vG02F0RmPGZAeg2BjYq8bCL1y5MzyXj7RnZy364+c6dkJBtS+Tnfa85D5CGBZoR1OvTzIgJLD5FJKz3yUwfci1Sh6QLlV7jgSSXXMqreXbKLxPOJT1H7oQ3Gqz7uTqqyQD4wTWA8UIvgj9DGL6bW72KmJXT/5A7P9IpAG8G670XWeBDX5sSvFkv/XNRYblv7mArywgCOurSQFxGYfiTWwHrZ1QaJ7QzzOb3dR7Umr1wCwPfIIDGYxyBEVbWVXkTs5dQXAcqbcA4KCsTh77o+7gEcZEJmx5wsM37ApAxcQxm0BQ4WUcuzyyPtAR+22aCXwzwqIPKIXLYg5iTIt+dPP1dFvvjQVKkuIrDkEFm9ocZ+GrTXPvxM59z8bS6nMYjr4CtrY53EAxIYfuM3uHrtA3DU6zRwA0Q8dfZXWNfMeVxEhmve+feXVKrGR9KouojAkkViFcaPBuMVuNYGUUmMCxteecWAfz/d/iE7se99tn/nFq2m7uB4S6uuZCcrZw0iMyz6482vaXfMvNe8p+Utc2zbnu1vqlSMbZbqakr7+iACSw6RYSddQleC4INVxmtpWk1GIrBkk1gN6/ONXUdXg+CgupAe0Zilk6YoZIIAk8B41Rp/fECXdAuCNqqrJmvkRQos2WqswlJjc+lqZBbw/i/Kgq+LCCy9RFZrqLGnWC+bSFcjM+iwiKsh6xeCTMjkm5WNPcePn2/88REyKzNjLlYQeZECS6MaQwLsUpaCcW6EQdhgqa5WuhREYGknsgqLyO6mq5F4wM+1NIsOeiIwIjIiMiIuIjACERmBiIsIjEBERnADxu41EHERgRH8iQzOfvQfqzdek+mKxAakQzQYr5XknCcCIwQjs3qLyKg8KTqgpTOSkNdnpWaRCIwQhXkJVVZHqkyZ2lpvqa0muhxEYAR1ZFZnERlepXRFQmGDpbQa6FIQgRGIzBJDWmQiEoER9COzWjIzXc1DvBqJtIjACPqTWYVFZLXWK2vqDPlajZbKIp8WERgh4YRWk0NmNSlTaB0WWTVZCquR7jgRGCHdhFZmEZlNaFBs1Qk4dHQ2bc0hrCbK0SICIxBylVqFRWplOT+jJDebpOwXiKqdlBURGIEgg+DKrL9WWK982MRnwyQgh/e1W/9nggiKCIxAIBC0AXVkJRAIRGAEAoFABEYgEAhEYAQCgQiMQCAQiMAIBAKBCIxAIBCIwAgEAhEYgUAgEIERCAQCERiBQCACIxAIBCIwAoFAIAIjEAiEM/j/AgwAH/jo3U5zdIUAAAAASUVORK5CYII=
    mediatype: image/png
  install:
    spec:
      clusterPermissions:
      - rules:
        - apiGroups:
          - ""
          resources:
          - serviceaccounts
          - namespaces
          - pods
          verbs:
          - '*'
        - apiGroups:
          - authentication.k8s.io
          resources:
          - tokenreviews
          verbs:
          - create
        - apiGroups:
          - authorization.k8s.io
          resources:
          - subjectaccessreviews
          verbs:
          - create
        - apiGroups:
          - rbac.authorization.k8s.io
          resources:
          - clusterroles
          - clusterrolebindings
          - rolebindings
          verbs:
          - create
          - delete
          - patch
          - get
          - list
        - apiGroups:
          - servicecatalog.k8s.io
          resources:
          - clusterservicebrokers
          - servicebrokers
          verbs:
          - '*'
        - apiGroups:
          - networking.k8s.io
          resources:
          - networkpolicies
          verbs:
          - create
          - delete
          - list
          - get
        - apiGroups:
          - authorization.openshift.io
          resources:
          - subjectrulesreview
          verbs:
          - create
        - apiGroups:
          - network.openshift.io
          - ""
          resources:
          - clusternetworks
          - netnamespaces
          verbs:
          - get
          - update
          - list
        - apiGroups:
          - image.openshift.io
          - ""
          resources:
          - images
          verbs:
          - get
          - list
        - apiGroups:
          - osb.openshift.io
          resources:
          - '*'
          verbs:
          - '*'
        - apiGroups:
          - automationbroker.io
          resources:
          - '*'
          verbs:
          - '*'
        - nonResourceURLs:
          - /osb
          - /osb/*
          verbs:
          - get
          - post
          - put
          - patch
          - delete
        serviceAccountName: openshift-ansible-service-broker-operator
      deployments:
      - name: openshift-ansible-service-broker-operator
        spec:
          replicas: 1
          selector:
            matchLabels:
              name: openshift-ansible-service-broker-operator-alm-owned
          template:
            metadata:
              labels:
                name: openshift-ansible-service-broker-operator-alm-owned
              name: openshift-ansible-service-broker-operator-alm-owned
            spec:
              containers:
              - env:
                - name: IMAGE
                  value: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker:v4.2.0
                - name: OPERATOR_NAME
                  value: openshift-ansible-service-broker-operator
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
                - name: WATCH_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
                image: image-registry.openshift-image-registry.svc:5000/openshift/ose-ansible-service-broker-operator:v4.2.0
                imagePullPolicy: IfNotPresent
                name: openshift-ansible-service-broker-operator
              serviceAccountName: openshift-ansible-service-broker-operator
      permissions:
      - rules:
        - apiGroups:
          - ""
          resources:
          - namespaces
          verbs:
          - get
        - apiGroups:
          - ""
          resources:
          - pods
          - configmaps
          - secrets
          - services
          - services/finalizers
          verbs:
          - '*'
        - apiGroups:
          - apps
          resources:
          - deployments
          - deployments/finalizers
          - replicasets
          verbs:
          - '*'
        - apiGroups:
          - route.openshift.io
          resources:
          - routes
          verbs:
          - '*'
        - apiGroups:
          - apps.openshift.io
          resources:
          - deploymentconfigs
          verbs:
          - '*'
        - apiGroups:
          - monitoring.coreos.com
          resources:
          - servicemonitors
          verbs:
          - '*'
        serviceAccountName: openshift-ansible-service-broker-operator
    strategy: deployment
  installModes:
  - supported: true
    type: OwnNamespace
  - supported: true
    type: SingleNamespace
  - supported: false
    type: MultiNamespace
  - supported: false
    type: AllNamespaces
  keywords:
  - ansible
  - automation
  - broker
  - open service broker
  labels:
    alm-status-descriptors: openshiftansibleservicebroker.v4.2.0
    operated-by: openshiftansibleservicebroker
  links:
  - name: Blog
    url: http://automationbroker.io/
  - name: YouTube
    url: https://www.youtube.com/channel/UC04eOMIMiV06_RSZPb4OOBw
  - name: Source Code
    url: https://github.com/openshift/ansible-service-broker/
  maintainers:
  - email: ansible-service-broker
    name: Red Hat, Inc.
  maturity: stable
  provider:
    name: Red Hat, Inc.
  selector:
    matchLabels:
      operated-by: openshiftansibleservicebroker
  version: 4.2.0
status:
  certsLastUpdated: null
  certsRotateAt: null
  conditions:
  - lastTransitionTime: 2019-09-09T12:44:16Z
    lastUpdateTime: 2019-09-09T12:44:16Z
    message: requirements not yet checked
    phase: Pending
    reason: RequirementsUnknown
  - lastTransitionTime: 2019-09-09T12:44:16Z
    lastUpdateTime: 2019-09-09T12:44:16Z
    message: one or more requirements couldn't be found
    phase: Pending
    reason: RequirementsNotMet
  lastTransitionTime: 2019-09-09T12:44:16Z
  lastUpdateTime: 2019-09-09T12:44:16Z
  message: one or more requirements couldn't be found
  phase: Pending
  reason: RequirementsNotMet
  requirementStatus:
  - group: operators.coreos.com
    kind: ClusterServiceVersion
    message: CSV missing minimum kube version specification
    name: openshiftansibleservicebroker.v4.2.0
    status: NotPresent
    version: v1alpha1
  - group: apiextensions.k8s.io
    kind: CustomResourceDefinition
    message: CRD is present and Established condition is true
    name: automationbrokers.osb.openshift.io
    status: Present
    uuid: 06047c70-d2a9-11e9-9214-0a5191f3ea26
    version: v1beta1
  - group: apiextensions.k8s.io
    kind: CustomResourceDefinition
    message: CRD is present and Established condition is true
    name: bundlebindings.automationbroker.io
    status: Present
    uuid: 060703a5-d2a9-11e9-9214-0a5191f3ea26
    version: v1beta1
  - group: apiextensions.k8s.io
    kind: CustomResourceDefinition
    message: CRD is present and Established condition is true
    name: bundleinstances.automationbroker.io
    status: Present
    uuid: 0609a894-d2a9-11e9-9214-0a5191f3ea26
    version: v1beta1
  - group: apiextensions.k8s.io
    kind: CustomResourceDefinition
    message: CRD is present and Established condition is true
    name: bundles.automationbroker.io
    status: Present
    uuid: 0605a572-d2a9-11e9-9214-0a5191f3ea26
    version: v1beta1
  - group: ""
    kind: ServiceAccount
    message: Service account does not exist
    name: openshift-ansible-service-broker-operator
    status: NotPresent
    version: v1

Comment 11 Fabian von Feilitzsch 2019-09-09 19:36:20 UTC
Ok, so looking into it more, it seems that the cause of the error for the ASBO was the addition of the `OperatorHub` (in disbale_redhat_operator.yaml) and `OperatorSource` (in OP.yaml) to the Ansible Service Broker Operator bundle in the QE app registry. The template service broker operator does not have those resources added to the bundle, and therefore works fine. Those resources are not resources that OLM can resolve, so the InstallPlan is stuck in a failed state with the condition: 

 conditions:
    - lastTransitionTime: '2019-09-09T18:14:17Z'
      lastUpdateTime: '2019-09-09T18:14:17Z'
      message: the InstallPlan contains invalid data
      reason: InstallComponentFailed
      status: 'False'
      type: Installed


Removing those resources from the bundle should resolve the issue, and if those resources are necessary I believe they will need to be created separately.

Comment 12 Fabian von Feilitzsch 2019-09-09 20:09:54 UTC
Moving back to ON_QA, with those resources removed from the bundle the initial fix should work. Does anyone know how/why they were added? Just want to make sure there isn't an issue in the process that will cause this to crop up again.

Comment 13 Cuiping HUO 2019-09-10 04:37:58 UTC
@Fabian,
I tried to delete the  `OperatorHub` and `OperatorSource` addition in ASB InstallPlan, but find it recovered automatically. So how do you removing?

I use:
$ oc edit ip install-5mrdd -n openshift-ansible-service-broker 
to delete below
- resolving: openshiftansibleservicebroker.v4.2.0
    resource:
      group: operators.coreos.com
      kind: OperatorSource
      manifest: |
        {"apiVersion":"operators.coreos.com/v1","kind":"OperatorSource","metadata":{"name":"qe-app-registry","namespace":"openshift-marketplace"},"spec":{"authorizationToken":{"secretName":"qesecret"},"endpoint":"https://quay.io/cnr","registryNamespace":null,"type":"appregistry"}}
      name: qe-app-registry
      sourceName: qe-app-registry
      sourceNamespace: openshift-marketplace
      version: v1
    status: Unknown

- resolving: openshiftansibleservicebroker.v4.2.0
    resource:
      group: config.openshift.io
      kind: OperatorHub
      manifest: |
        {"apiVersion":"config.openshift.io/v1","kind":"OperatorHub","metadata":{"name":"cluster"},"spec":{"disableAllDefaultSources":false,"sources":[{"disabled":true,"name":"redhat-operators"}]}}
      name: cluster
      sourceName: qe-app-registry
      sourceNamespace: openshift-marketplace
      version: v1
    status: Unknown

Comment 14 Cuiping HUO 2019-09-10 09:36:30 UTC
Verified.
clusterversion: 4.2.0-0.nightly-2019-09-10-014843

$ oc get opsrc  -n openshift-marketplace
NAME                  TYPE          ENDPOINT              REGISTRY              DISPLAYNAME           PUBLISHER   STATUS      
qe-app-registry       appregistry   https://quay.io/cnr   aosqe42                                                 Succeeded   The object has been successfully reconciled   4h8m
$ oc get packagemanifest openshiftansibleservicebroker -n openshift-marketplace -o jsonpath='{range .status}{.channels[0].currentCSV}{"\n"}{end}'
openshiftansibleservicebroker.v4.2.0

$ oc get packagemanifest openshifttemplateservicebroker -n openshift-marketplace -o jsonpath='{range .status}{.channels[0].currentCSV}{"\n"}{end}'
openshifttemplateservicebrokeroperator.v4.2.0

$ oc get clusterservicebroker
NAME                      URL                                                                                         STATUS   AGE
ansible-service-broker    https://asb.openshift-ansible-service-broker.svc:1338/osb/                                  Ready    1h
template-service-broker   https://apiserver.openshift-template-service-broker.svc:443/brokers/template.openshift.io   Ready    4h
$ oc get po -n openshift-ansible-service-broker
NAME                                                         READY   STATUS      RESTARTS   AGE
asb-1-deploy                                                 0/1     Completed   0          78m
asb-1-xsd72                                                  2/2     Running     0          77m
openshift-ansible-service-broker-operator-567bf4d6f6-r9dmd   1/1     Running     0          81m
$ oc get po -n openshift-template-service-broker
NAME                                                          READY   STATUS      RESTARTS   AGE
apiserver-1-deploy                                            0/1     Completed   0          4h2m
apiserver-1-kplxv                                             1/1     Running     0          4h
openshift-template-service-broker-operator-6546d5bf55-ghdsd   1/1     Running     0          4h3m

Comment 15 errata-xmlrpc 2019-10-16 06:39:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922


Note You need to log in before you can comment on or make changes to this bug.