Bug 1667363 - config map can not be updated If ASB is deployed by operator
Summary: config map can not be updated If ASB is deployed by operator
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Service Broker
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.1.0
Assignee: jkim
QA Contact: Zihan Tang
URL:
Whiteboard:
Depends On: 1668534
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-18 09:19 UTC by Zihan Tang
Modified: 2019-06-04 10:42 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:42:02 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:42:09 UTC

Description Zihan Tang 2019-01-18 09:19:51 UTC
Description of problem:
If ASB is deployed by ASB operator, cm broker-config can not update successfully.

Version-Release number of selected component (if applicable):
openshift-install v0.10.0
Cluster version is 4.0.0-0.1
OLM version: 0.8.1  git commit: 54a541d

How reproducible:
always

Steps to Reproduce:
1. deploy asb operator and asb using file: https://github.com/fusor/catbrokers4
2. edit registry in cm broker-config 
3.

Actual results:
cm can not update successfully. will rollback automatically.

Expected results:
Provide some ways to update cm 

Additional info:
we have tried to stop the asb operators and some other ways, sometime it brokes the asb pod, and still not update cm successfully.

Comment 1 Zihan Tang 2019-01-18 09:21:23 UTC
This bug blocks current test run, so add testbloker keyword.

Comment 2 Zhang Cheng 2019-01-18 10:51:07 UTC
We noticed the configmap of automation broker is watching by operator. And no method to change broker config at present. We tried to stop (or remove) broker operator, but asb pod crash since didn't find relate resource. 

So, we lLeave TestBlocker in here to block related TCs in current test run, and will remove it while get fix or a workaround. Thanks.


# oc rsh automation-broker-operator-66b644bfb5-xjn69
sh-4.2$ ps -ef | grep ansible-operator
ansible+      1      0  1 07:30 ?        00:01:00 /usr/local/bin/ansible-operator run ansible --watches-file=/opt/ansible/watches.yaml

sh-4.2$ cat watches.yaml
---

- name: AutomationBroker
  version: v1alpha1
  group: automationbroker.io
  kind: AutomationBroker
  playbook: /opt/ansible/deploy.yml
  watchDependentResources: false
  finalizer:
    name: finalizer.automationbroker.io
    vars:
      action: deprovision
      broker_destroy_shared_resources: true

sh-4.2$ cat /opt/ansible/deploy.yml
---

- name: automation-broker-operator
  hosts: localhost
  become: false
  gather_facts: false
  connection: local
  vars:
    state: present
    broker_name: "{{ meta.name }}"
    broker_namespace: "{{ meta.namespace }}"
    broker_image: "{{ lookup('env', 'IMAGE') }}"
  tasks:
    - name: Validation
      assert:
        that: broker_name and broker_namespace and broker_image
        fail_msg: >
          broker_name broker_namespace broker_image must be defined
          The IMAGE environment variable must provide the broker's image name
          and should be specified in the operator's deployment specification.
    - name: Run automation-broker role
      include_role:
        name: automation-broker

sh-4.2$ cat roles/automation-broker/vars/configmap.yml
---

broker_config_map:
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: broker-config
    namespace: '{{ broker_namespace }}'
    labels:
      app: '{{ broker_name }}'
  data:
    broker-config: '{{ broker_config | to_nice_yaml }}'

Comment 3 Zhang Cheng 2019-01-18 11:00:05 UTC
John, 

Could developer provide a workaround to change broker config at present? We can accept stop watching by broker operator until get formal solution since we are focus on regression test and ensuring old features/functions still work fine in OCP4.0, especially in operator of brokers is not shipped.

Comment 4 Erik Nelson 2019-01-18 13:22:20 UTC
It should be possible to do this with the current images, I'm confirming that this morning and will update this bz with the results and advice.

Comment 5 Erik Nelson 2019-01-18 15:56:54 UTC
I've tested and confirmed that the currently published ASB operator image supports broker configuration via the CR. You need to add a complete broker configuration to the CR under the "brokerConfig" variable. This will override the operator's defaults and the operator will simply take that configuration and create a ConfigMap for the broker. Additionally, it also supports changes. If you would like to change any configuration values for an operator managed broker, you should edit the broker config that is in corresponding CR. The operator will recognize that the configuration has changed, update the ConfigMap, and recreate the broker's pod with the new config mounted in.

Here is an example CR that contains a broker_config override: https://github.com/eriknelson/catbrokers4/blob/add-config-example/files/asb/05-asb-cr.yaml#L8

We are in the process of merging this to the catbrokers4 repo so that the playbook will use it.

Comment 6 Zhang Cheng 2019-01-19 02:08:04 UTC
Erik, Thanks for your quickly response.
We will checking

Comment 7 Zihan Tang 2019-01-21 07:01:59 UTC
At present, we can not deploy any operator successfully due to bug https://bugzilla.redhat.com/show_bug.cgi?id=1666225
we'll try when #bug 1666225 fixed or find workaround.

Comment 9 Zhang Cheng 2019-01-30 23:56:18 UTC
Yes. But we cannot verify it since it is blocking by another bug https://bugzilla.redhat.com/show_bug.cgi?id=1668534

So, we marked 'Depends On: 1668534'.

Comment 10 Erik Nelson 2019-02-01 20:30:09 UTC
The broker's operator has been updated with a limited set of supported configuration options that should be set in the broker's CR. All of the supported options are documented here: https://github.com/openshift/ansible-service-broker/blob/master/docs/operator.md

Manually editing the broker's config directly, in addition to setting the broker's config directly, are both no longer supported. If you must edit the broker's configuration directly, you will need to scale down the broker's operator by deleting the CSV + subscription in the operator's namespace. Then you will be able to edit the configmap directly without the operator overwriting your changes.

The updated operator image has been published upstream to docker.io/automationbroker/automation-broker-operator.

Comment 12 Zihan Tang 2019-02-19 10:24:31 UTC
Verified.
asb operator: docker.io/automationbroker/osb-operators
asb version: 1.4.4

Then cm can be updated in the following scenario:
1. recreate the automationbroker CR and add the registries etc. config in the CR.
2. edit automationbroker CR , add the config in 'spec' like: 
  brokerImage: docker.io/zitangbj/ose-ansible-service-broker:v4.0
  registries:
    - type: openshift
      name: aws
      url:  https://registry.reg-aws.openshift.com:443 
      tag:  v4.0
      images:
        - openshift/mediawiki-apb
        - openshift/postgresql-apb
        - openshift/mariadb-apb
....
   when the automationbroker work well, it will trigger update asb pod automatically, but asb operator pod often produce some error after running for some time and can not monitor the update or deletion of CR. I'll open another bug to trace it.
3. delete csv and subscription, then can edit broker-config cm directly as before v3.11

Comment 15 errata-xmlrpc 2019-06-04 10:42:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.