Bug 1691230 - Too many wrong logs "Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-x-x" in EO pod log.
Summary: Too many wrong logs "Reusing existing PVC: elasticsearch-elasticsearch-client...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.1.0
Assignee: ewolinet
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-21 08:26 UTC by Qiaoling Tang
Modified: 2019-06-04 10:46 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:46:16 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 None None None 2019-06-04 10:46:23 UTC

Description Qiaoling Tang 2019-03-21 08:26:20 UTC
Description of problem:

There are too many "Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-x-x" in EO pod log when pvc enabled. These logs should only generated when the es pods redeployed, but now the logs generated frequently, and in fact none of the es pods is redeployed.

$ oc get pod -n openshift-operators
NAME                                      READY   STATUS    RESTARTS   AGE
elasticsearch-operator-5b4977987d-d4jxt   1/1     Running   0          7m58s

$ oc logs -n openshift-operators elasticsearch-operator-5b4977987d-d4jxt
time="2019-03-21T05:38:16Z" level=info msg="Go Version: go1.10.3"
time="2019-03-21T05:38:16Z" level=info msg="Go OS/Arch: linux/amd64"
time="2019-03-21T05:38:16Z" level=info msg="operator-sdk Version: 0.0.7"
time="2019-03-21T05:38:16Z" level=info msg="Watching logging.openshift.io/v1alpha1, Elasticsearch, , 5000000000"
time="2019-03-21T05:39:51Z" level=info msg="Creating new PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:39:51Z" level=info msg="Creating new PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:40:24Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:40:25Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:40:49Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:40:49Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:41:10Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:41:10Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:41:34Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:41:34Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:41:45Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:41:45Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:41:55Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:41:55Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:42:05Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:42:05Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:42:15Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:42:15Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:42:24Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:42:25Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:42:34Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:42:34Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:42:44Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:42:44Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:42:54Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:42:54Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:43:04Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:43:04Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:43:13Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:43:14Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:43:23Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:43:23Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:43:33Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:43:33Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:43:43Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:43:43Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:43:53Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:43:53Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:44:02Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:44:03Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:44:12Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:44:12Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:44:22Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:44:22Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:44:32Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:44:32Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:44:42Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:44:42Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:44:51Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:44:52Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:45:01Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:45:01Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:45:11Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:45:11Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:45:21Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:45:21Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:45:31Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:45:31Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:45:40Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:45:41Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:45:50Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:45:50Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"
time="2019-03-21T05:46:00Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-1"
time="2019-03-21T05:46:00Z" level=info msg="Reusing existing PVC: elasticsearch-elasticsearch-clientdatamaster-0-2"

$ oc get pod
NAME                                                  READY   STATUS    RESTARTS   AGE
cluster-logging-operator-78595b68c7-mkrmx             1/1     Running   0          6m38s
elasticsearch-clientdatamaster-0-1-6db4f44f6f-ngkrz   2/2     Running   0          6m20s
elasticsearch-clientdatamaster-0-2-665b59bdc9-6pndv   2/2     Running   0          6m5s
fluentd-578n2                                         1/1     Running   0          6m14s

$ oc get pvc
NAME                                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
elasticsearch-elasticsearch-clientdatamaster-0-1   Bound    pvc-bf558bad-4b9b-11e9-91cf-02b29c8e3c7e   10Gi       RWO            gp2            7m48s
elasticsearch-elasticsearch-clientdatamaster-0-2   Bound    pvc-bf91c0de-4b9b-11e9-91cf-02b29c8e3c7e   10Gi       RWO            gp2            7m48s

Version-Release number of selected component (if applicable):
image: quay.io/openshift/origin-elasticsearch-operator:latest
imageID: quay.io/openshift/origin-elasticsearch-operator@sha256:c30715c23f4dc6f04c22040df5547a738c13f42c6e79a27225f597ce6012a2a3
4.0.0-0.nightly-2019-03-20-153904

How reproducible:
Always

Steps to Reproduce:
1.Deploy logging via community-operators with pvc enabled
2.check EO pod logs
3.

Actual results:


Expected results:


Additional info:

Comment 2 Qiaoling Tang 2019-03-26 05:23:25 UTC
Verified in quay.io/openshift/origin-elasticsearch-operator@sha256:8e7a748802fc284162f5dadf7cdfd9ae2adfb72b6f9682d3b91ee87024aa0a76

Comment 4 errata-xmlrpc 2019-06-04 10:46:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.