Bug 1688647

Description of problem:
Deploy logging via community-operators, wait for all pods running, set curator cronjob schedule time to: */5 * * * * , then wait for a while, the curator pod in error status, check curator pod log, there is an error message: curator.exceptions.ConfigurationError: Configuration: Client Configuration: Location: full configuration dictionary: Bad Value: "{'certificate': '${ES_CA}', 'client_cert': '${ES_CLIENT_CERT}', 'hosts': ['${ES_HOST}'], 'timeout': '${CURATOR_TIMEOUT}', 'use_ssl': True, 'master_only': False, 'port': '${ES_PORT}', 'ssl_no_validate': False, 'client_key': '${ES_CLIENT_KEY}'}", not a valid value for dictionary value @ data['client']['port']. Check configuration file.

$ oc get pod
NAME                                                  READY   STATUS    RESTARTS   AGE
cluster-logging-operator-5bc59f67bf-c6stq             1/1     Running   0          31m
curator-1552546200-9wlfw                              0/1     Error     0          3m52s
elasticsearch-clientdatamaster-0-1-555888f475-jzdkd   2/2     Running   0          6m57s

$ oc logs curator-1552546200-9wlfw
2019-03-14 06:50:14,269 INFO	Found curator configuration in [/etc/curator/settings/config.yaml]
2019-03-14 06:50:14,278 INFO	Converting config file.
/usr/lib/python2.7/site-packages/curator/utils.py:53: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  cfg = yaml.load(raw)
No handlers could be found for logger "curator.validators.SchemaCheck"
Traceback (most recent call last):
  File "/usr/bin/curator", line 9, in <module>
    load_entry_point('elasticsearch-curator==5.2.0', 'console_scripts', 'curator')()
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib64/python2.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/curator/cli.py", line 211, in cli
    run(config, action_file, dry_run)
  File "/usr/lib/python2.7/site-packages/curator/cli.py", line 106, in run
    client_args = process_config(config)
  File "/usr/lib/python2.7/site-packages/curator/config_utils.py", line 45, in process_config
    config = test_config(yaml_file)
  File "/usr/lib/python2.7/site-packages/curator/config_utils.py", line 19, in test_config
    'Client Configuration', 'full configuration dictionary').result()
  File "/usr/lib/python2.7/site-packages/curator/validators/schemacheck.py", line 68, in result
    self.test_what, self.location, self.badvalue, self.error)
curator.exceptions.ConfigurationError: Configuration: Client Configuration: Location: full configuration dictionary: Bad Value: "{'certificate': '${ES_CA}', 'client_cert': '${ES_CLIENT_CERT}', 'hosts': ['${ES_HOST}'], 'timeout': '${CURATOR_TIMEOUT}', 'use_ssl': True, 'master_only': False, 'port': '${ES_PORT}', 'ssl_no_validate': False, 'client_key': '${ES_CLIENT_KEY}'}", not a valid value for dictionary value @ data['client']['port']. Check configuration file.

$ oc get cm curator -o yaml
apiVersion: v1
data:
  actions.yaml: |
    # ---
    # Remember, leave a key empty if there is no value.  None will be a string,
    # not a Python "NoneType"
    #
    # Also remember that all examples have 'disable_action' set to True.  If you
    # want to use this action as a template, be sure to set this to False after
    # copying it.
    # actions:
    #   1:
    #     action: delete_indices
    #     description: >-
    #       Delete .operations indices older than 30 days.
    #       Ignore the error if the filter does not
    #       result in an actionable list of indices (ignore_empty_list).
    #       See https://www.elastic.co/guide/en/elasticsearch/client/curator/5.2/ex_delete_indices.html
    #     options:
    #       # Swallow curator.exception.NoIndices exception
    #       ignore_empty_list: True
    #       # In seconds, default is 300
    #       timeout_override: ${CURATOR_TIMEOUT}
    #       # Don't swallow any other exceptions
    #       continue_if_exception: False
    #       # Optionally disable action, useful for debugging
    #       disable_action: False
    #     # All filters are bound by logical AND
    #     filters:
    #     - filtertype: pattern
    #       kind: regex
    #       value: '^\.operations\..*$'
    #       exclude: False
    #     - filtertype: age
    #       # Parse timestamp from index name
    #       source: name
    #       direction: older
    #       timestring: '%Y.%m.%d'
    #       unit: days
    #       unit_count: 30
    #       exclude: False
  config.yaml: |
    # Logging example curator config file

    # uncomment and use this to override the defaults from env vars
    #.defaults:
    #  delete:
    #    days: 30

    # to keep ops logs for a different duration:
    #.operations:
    #  delete:
    #    weeks: 8

    # example for a normal project
    #myapp:
    #  delete:
    #    weeks: 1
  curator5.yaml: "---\nclient:\n  hosts:\n  - ${ES_HOST}\n  port: ${ES_PORT}\n  use_ssl:
    True\n  certificate: ${ES_CA}\n  client_cert: ${ES_CLIENT_CERT}\n  client_key:
    ${ES_CLIENT_KEY}\n  ssl_no_validate: False\n  timeout: ${CURATOR_TIMEOUT}\n  master_only:
    False\nlogging:\n  loglevel: ${CURATOR_LOG_LEVEL}\n  logformat: default\n  blacklist:
    ['elasticsearch', 'urllib3']\n  \n"
kind: ConfigMap
metadata:
  creationTimestamp: 2019-03-14T06:46:59Z
  name: curator
  namespace: openshift-logging
  ownerReferences:
  - apiVersion: logging.openshift.io/v1alpha1
    controller: true
    kind: ClusterLogging
    name: customresourcefluentd
    uid: f38fd146-4624-11e9-88a3-0afa9a7871ea
  resourceVersion: "168406"
  selfLink: /api/v1/namespaces/openshift-logging/configmaps/curator
  uid: f725bfdd-4624-11e9-90f1-065f5fd6ebc0

Version-Release number of selected component (if applicable):
4.0.0-0.nightly-2019-03-13-233958

How reproducible:
Always

Steps to Reproduce:
1.Deploy logging via community operators
2.wait for curator pod start
3.

Actual results:


Expected results:


Additional info: