Bug 1694522 - Upgrade ES via OLM failed because of "Unsupported change to UUIDs made"
Summary: Upgrade ES via OLM failed because of "Unsupported change to UUIDs made"
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.1.0
Assignee: ewolinet
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-01 02:33 UTC by Qiaoling Tang
Modified: 2019-06-04 10:46 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:46:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:46:47 UTC

Description Qiaoling Tang 2019-04-01 02:33:03 UTC
Description of problem:
Upgrade the logging cluster via OLM, the ES pod isn't upgraded, error message in elasticsearch-operator pod log:
time="2019-04-01T01:57:03Z" level=error msg="error syncing key (openshift-logging/elasticsearch): Failed to reconcile Elasticsearch deployment spec: Unsupported change to UUIDs made: Previously used GenUUID \"79gc79ge\" is no longer found in Spec.Nodes"

elasticsearch CR:
apiVersion: v1
items:
- apiVersion: logging.openshift.io/v1
  kind: Elasticsearch
  metadata:
    creationTimestamp: 2019-04-01T01:10:23Z
    generation: 1
    name: elasticsearch
    namespace: openshift-logging
    ownerReferences:
    - apiVersion: logging.openshift.io/v1
      controller: true
      kind: ClusterLogging
      name: instance
      uid: daacbc46-541a-11e9-872e-0299e54d8204
    resourceVersion: "49196"
    selfLink: /apis/logging.openshift.io/v1/namespaces/openshift-logging/elasticsearches/elasticsearch
    uid: ece23de6-541a-11e9-872e-0299e54d8204
  spec:
    managementState: Managed
    nodeSpec:
      image: quay.io/openshift/origin-logging-elasticsearch5:v4.0
      resources:
        limits:
          cpu: "1"
          memory: 4Gi
        requests:
          cpu: 200m
          memory: 1Gi
    nodes:
    - nodeCount: 2
      resources:
        limits:
          cpu: "1"
          memory: 4Gi
        requests:
          cpu: 200m
          memory: 1Gi
      roles:
      - client
      - data
      - master
      storage:
        size: 10Gi
        storageClassName: gp2
    redundancyPolicy: SingleRedundancy
  status:
    clusterHealth: green
    conditions:
    - lastTransitionTime: 2019-04-01T01:57:03Z
      message: Previously used GenUUID "79gc79ge" is no longer found in Spec.Nodes
      reason: Invalid Spec
      status: "True"
      type: InvalidUUID

Other pods are upgraded successfully. The differences between these two csv versions is the container image tag, in 4.2.0, the tag is "v4.0"
$ oc get csv
NAME                            DISPLAY                  VERSION   REPLACES                        PHASE
clusterlogging.v4.2.0           Cluster Logging          4.2.0     clusterlogging.v4.1.0           Succeeded
elasticsearch-operator.v4.2.0   Elasticsearch Operator   4.2.0     elasticsearch-operator.v4.1.0   Succeeded

Version-Release number of selected component (if applicable):
quay.io/openshift/origin-elasticsearch-operator@sha256:03cf12040145c6c862726c732942cda0f8d3b3bd8d9fedf583911be10b4e7756

How reproducible:
Always

Steps to Reproduce:
1. Deploy logging via OLM
2. upgrade logging via OLM
the new csv files are:
https://raw.githubusercontent.com/QiaolingTang/logging/master/cluster-logging-bundle.yaml
https://raw.githubusercontent.com/QiaolingTang/logging/master/elasticsearch-operator-bundle.yaml
3.

Actual results:
The es pod isn't upgraded.

Expected results:


Additional info:

Comment 1 Qiaoling Tang 2019-04-01 02:34:27 UTC
$ oc get deploy elasticsearch-cdm-79gc79ge-1 -o yaml |grep image
        image: quay.io/openshift/origin-logging-elasticsearch5:latest
        imagePullPolicy: IfNotPresent
        image: quay.io/openshift/origin-oauth-proxy:latest
        imagePullPolicy: IfNotPresent
$ oc get deploy -n openshift-operators -oyaml |grep IMAGE -A 1
          - name: PROXY_IMAGE
            value: quay.io/openshift/origin-oauth-proxy:v4.0

Comment 4 Qiaoling Tang 2019-04-09 02:16:52 UTC
Verified in latest CLO image:
quay.io/openshift/origin-cluster-logging-operator@sha256:949ee74661a3bac7d08084d01ce1375ff51a04f97d28ff59d7e35f49e5065a15

Comment 6 errata-xmlrpc 2019-06-04 10:46:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.