Bug 1691148 - Adding one data node resulting in all ES new deployment
Summary: Adding one data node resulting in all ES new deployment
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.1.0
Assignee: ewolinet
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-21 01:24 UTC by Anping Li
Modified: 2019-06-04 10:46 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:46:16 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:46:23 UTC

Description Anping Li 2019-03-21 01:24:03 UTC
Description of problem:

I want to add one data node in the logging cluster.  I added one item in the CRD elasticsearch.  As this item was added prior the existing one [1].  The elasticsearch operator create new deployment for all ES nodes[2]. 

I don't realized the nodes are designed as json array until I observed this.  the elasticsearch deployment name rely on the sequence in the array.  I raise this issue to discuss if we imporve the way to deploy elasticsearch. For example: add a mandatory name for each type of node item [3].

[1]
  nodes:
  - nodeCount: 1
    roles:
    - data
  - nodeCount: 3
    roles:
    - client
    - data
    - master
[2]
deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
deployment.extensions/elasticsearch-clientdatamaster-1-1
deployment.extensions/elasticsearch-clientdatamaster-1-2
deployment.extensions/elasticsearch-clientdatamaster-1-3
deployment.extensions/elasticsearch-data-0-1


[3]
  nodes:
  - nodeCount: 1
    roles:
    - data
    name: all-in-one
  - nodeCount: 3
    roles:
    - client
    - data
    - master
    name: data

Version-Release number of selected component (if applicable):
v4.x

How reproducible:
always

Steps to Reproduce:
1. Deploy clusterlogging via clusterlogging operators
2. Create logging CRD
   oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/logging/clusterlogging/storageclass_name.yaml
3. there are three ES deployment

deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
4. Disable the clusterlogging operators
   set replicas=0 or  managementState=Unmanaged
5. Add one data node as following 
  nodes:
  - nodeCount: 1
    roles:
    - data
  - nodeCount: 3
    roles:
    - client
    - data
    - master

6.check the deployment
  oc get deployment

Actual results:
The old resource wasn't deleted, the new resource are created.

deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
deployment.extensions/elasticsearch-clientdatamaster-1-1
deployment.extensions/elasticsearch-clientdatamaster-1-2
deployment.extensions/elasticsearch-clientdatamaster-1-3
deployment.extensions/elasticsearch-data-0-1

oc get pvc -o name
NAME                                              
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-0-1 
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-0-2
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-0-3
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-1-1
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-1-2
persistentvolumeclaim/elasticsearch-elasticsearch-clientdatamaster-1-3
persistentvolumeclaim/elasticsearch-elasticsearch-data-0-1

Expected results:
Only one data node deployment are added.

deployment.extensions/elasticsearch-clientdatamaster-0-1
deployment.extensions/elasticsearch-clientdatamaster-0-2
deployment.extensions/elasticsearch-clientdatamaster-0-3
deployment.extensions/elasticsearch-data-0-1

Comment 1 ewolinet 2019-03-21 16:40:05 UTC
The ordering in the CR is what is causing this..

I can recreate this issue by inserting a node definition at the top of the nodes section, however if I append it this doesn't happen.

I will try to figure out a way to be defensive against this.

Comment 4 Anping Li 2019-04-01 02:15:55 UTC
The field genUUID is generated for each node spec, the deployment name are created following this field.  the node data only node is added as expected.  Move to verified.

deployment.extensions/elasticsearch-cdm-mdkstjvg-1
deployment.extensions/elasticsearch-cdm-mdkstjvg-2
deployment.extensions/elasticsearch-d-07p4ztr1-1

Comment 6 errata-xmlrpc 2019-06-04 10:46:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.