Moving to 4.2

Work around is to wait for the CLO to update the Elasticsearch CR and manually run the init script:

"oc exec -c elasticssearch $espod -- bash -c '$HOME/init.sh'"

(In reply to Jeff Cantrill from comment #2)
> Moving to 4.2
> 
> Work around is to wait for the CLO to update the Elasticsearch CR and
> manually run the init script:
> 
> "oc exec -c elasticssearch $espod -- bash -c '$HOME/init.sh'"

The workaround doesn't work, the index.number_of_replicas in template isn't changed after running the init script.

(In reply to Qiaoling Tang from comment #3)
> (In reply to Jeff Cantrill from comment #2)
> > Moving to 4.2
> > 
> > Work around is to wait for the CLO to update the Elasticsearch CR and
> > manually run the init script:
> > 
> > "oc exec -c elasticssearch $espod -- bash -c '$HOME/init.sh'"
> 
> The workaround doesn't work, the index.number_of_replicas in template isn't
> changed after running the init script.

I'm able to recreate this, but I think this issue is actually sourced from the fact that we use `sed -i` of the template files, so the first time it is run it will always have that value (not from the operator):

$ oc exec example-elasticsearch-cdm-vy14npva-1-55b6667ff5-5vxxv -c elasticsearch -- bash -c 'source /usr/share/java/elasticsearch/config/index_settings; echo $PRIMARY_SHARDS; echo $REPLICA_SHARDS'
2
1

$ oc exec example-elasticsearch-cdm-vy14npva-1-55b6667ff5-5vxxv -c elasticsearch -- bash -c 'cat $ES_HOME/index_templates/common.settings.project.template.json'
{
  "order": 5,
  "settings": {
    "index.refresh_interval": "5s",
    "index.number_of_replicas": 0,
    "index.number_of_shards": 1,
    "index.translog.flush_threshold_size": "256mb",
    "index.unassigned.node_left.delayed_timeout": "2m"
  },
  "template": "project*"

https://github.com/openshift/origin-aggregated-logging/pull/1604
to fix workaround

Workaround fix merged in, will need to wait for latest elasticsearch image to be published before it is available

Thanks for the comment.

The templates on the cluster are not updated after changing the redundancy policy from zeroredundancy to singleredundancy.

$ oc exec elasticsearch-cdm-y65z0agi-2-795b95c7fb-pq5rh -- es_util --query=_template/common.* |jq
Defaulting container name to elasticsearch.
Use 'oc describe pod/elasticsearch-cdm-y65z0agi-2-795b95c7fb-pq5rh -n openshift-logging' to see all of the containers in this pod.
{
  "common.settings.operations.orphaned.json": {
    "order": 5,
    "template": ".orphaned*",
    "settings": {
      "index": {
        "refresh_interval": "5s",
        "unassigned": {
          "node_left": {
            "delayed_timeout": "2m"
          }
        },
        "number_of_shards": "3",
        "translog": {
          "flush_threshold_size": "256mb"
        },
        "number_of_replicas": "0"
      }
    },
    "mappings": {},
    "aliases": {}
  },
  "common.settings.operations.template.json": {
    "order": 5,
    "template": ".operations*",
    "settings": {
      "index": {
        "refresh_interval": "5s",
        "unassigned": {
          "node_left": {
            "delayed_timeout": "2m"
          }
        },
        "number_of_shards": "3",
        "translog": {
          "flush_threshold_size": "256mb"
        },
        "number_of_replicas": "0"
      }
    },
    "mappings": {},
    "aliases": {}
  },
  "common.settings.project.template.json": {
    "order": 5,
    "template": "project*",
    "settings": {
      "index": {
        "refresh_interval": "5s",
        "unassigned": {
          "node_left": {
            "delayed_timeout": "2m"
          }
        },
        "number_of_shards": "3",
        "translog": {
          "flush_threshold_size": "256mb"
        },
        "number_of_replicas": "0"
      }
    },
    "mappings": {},
    "aliases": {}
  },
  "common.settings.kibana.template.json": {
    "order": 0,
    "template": ".kibana*",
    "settings": {
      "index": {
        "number_of_shards": "3",
        "number_of_replicas": "0"
      }
    },
    "mappings": {},
    "aliases": {}
  }
}

$ oc get cm elasticsearch -oyaml |grep index_settings -A 3
  index_settings: |2

    PRIMARY_SHARDS=3
    REPLICA_SHARDS=1

Images:
ose-logging-elasticsearch5-v4.2.0-201907071316
ose-cluster-logging-operator-v4.2.0-201907071316
ose-elasticsearch-operator-v4.2.0-201907071316

Per comment 11, move this bug to ASSIGNED.

I can recreate this with an image built from the master branch and see this fixed with https://github.com/openshift/elasticsearch-operator/pull/173

Both the confimap and template are updated when i changed the redundancyPolicy.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922