Bug 1691148

Summary: Adding one data node resulting in all ES new deployment
Product: OpenShift Container Platform Reporter: Anping Li <anli>
Component: LoggingAssignee: ewolinet
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.1.0CC: aos-bugs, ewolinet, rmeggins
Target Milestone: ---   
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:46:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Anping Li 2019-03-21 01:24:03 UTC
Description of problem:

I want to add one data node in the logging cluster.  I added one item in the CRD elasticsearch.  As this item was added prior the existing one [1].  The elasticsearch operator create new deployment for all ES nodes[2]. 

I don't realized the nodes are designed as json array until I observed this.  the elasticsearch deployment name rely on the sequence in the array.  I raise this issue to discuss if we imporve the way to deploy elasticsearch. For example: add a mandatory name for each type of node item [3].

[1]
  nodes:
  - nodeCount: 1
    roles:
    - data
  - nodeCount: 3
    roles:
    - client
    - data
    - master
[2]
deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
deployment.extensions/elasticsearch-clientdatamaster-1-1
deployment.extensions/elasticsearch-clientdatamaster-1-2
deployment.extensions/elasticsearch-clientdatamaster-1-3
deployment.extensions/elasticsearch-data-0-1


[3]
  nodes:
  - nodeCount: 1
    roles:
    - data
    name: all-in-one
  - nodeCount: 3
    roles:
    - client
    - data
    - master
    name: data

Version-Release number of selected component (if applicable):
v4.x

How reproducible:
always

Steps to Reproduce:
1. Deploy clusterlogging via clusterlogging operators
2. Create logging CRD
   oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/logging/clusterlogging/storageclass_name.yaml
3. there are three ES deployment

deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
4. Disable the clusterlogging operators
   set replicas=0 or  managementState=Unmanaged
5. Add one data node as following 
  nodes:
  - nodeCount: 1
    roles:
    - data
  - nodeCount: 3
    roles:
    - client
    - data
    - master

6.check the deployment
  oc get deployment

Actual results:
The old resource wasn't deleted, the new resource are created.

deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
deployment.extensions/elasticsearch-clientdatamaster-1-1
deployment.extensions/elasticsearch-clientdatamaster-1-2
deployment.extensions/elasticsearch-clientdatamaster-1-3
deployment.extensions/elasticsearch-data-0-1

oc get pvc -o name
NAME                                              
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-0-1 
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-0-2
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-0-3
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-1-1
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-1-2
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-1-3
persistentvolumeclaim/elasticsearch-elasticsearch-data-0-1

Expected results:
Only one data node deployment are added.

deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
deployment.extensions/elasticsearch-data-0-1

Comment 1 ewolinet 2019-03-21 16:40:05 UTC
The ordering in the CR is what is causing this..

I can recreate this issue by inserting a node definition at the top of the nodes section, however if I append it this doesn't happen.

I will try to figure out a way to be defensive against this.

Comment 4 Anping Li 2019-04-01 02:15:55 UTC
The field genUUID is generated for each node spec, the deployment name are created following this field.  the node data only node is added as expected.  Move to verified.

deployment.extensions/elasticsearch-cdm-mdkstjvg-1
deployment.extensions/elasticsearch-cdm-mdkstjvg-2
deployment.extensions/elasticsearch-d-07p4ztr1-1

Comment 6 errata-xmlrpc 2019-06-04 10:46:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758