Bug 1900873 - Frequent failure of the elastic delete and rollover jobs OCP 4.6.4 ES 4.6.0
Summary: Frequent failure of the elastic delete and rollover jobs OCP 4.6.4 ES 4.6.0
Keywords:
Status: CLOSED DUPLICATE of bug 1890838
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.6
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Jeff Cantrill
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-11-23 21:28 UTC by Dan Seals
Modified: 2020-11-23 21:35 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-23 21:35:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Dan Seals 2020-11-23 21:28:32 UTC
Customer seeing frequent failures of elasticsearch rollover and delete pods.

NAMESPACE                   NAME                                           READY  STATUS     RESTARTS  AGE
openshift-logging           cluster-logging-operator-555d665964-cqmhf      1/1    Running    0         4d
openshift-logging           curator-1606102200-lzczg                       0/1    Succeeded  0         7h56m
openshift-logging           elasticsearch-cdm-bxzkbe5l-1-7567bb475-lhmxj   2/2    Running    0         4d
openshift-logging           elasticsearch-cdm-bxzkbe5l-2-985f7c856-lf6zg   2/2    Running    0         4d
openshift-logging           elasticsearch-cdm-bxzkbe5l-3-657fb9d5fc-t5dt2  2/2    Running    0         4d
openshift-logging           elasticsearch-delete-app-1606130100-wmz4n      0/1    Succeeded  0         11m
openshift-logging           elasticsearch-delete-audit-1605896100-54mlp    0/1    Failed     0         2d
openshift-logging           elasticsearch-delete-audit-1606130100-wtbc4    0/1    Succeeded  0         11m
openshift-logging           elasticsearch-delete-infra-1605896100-nzndt    0/1    Failed     0         2d
openshift-logging           elasticsearch-delete-infra-1606130100-26fmp    0/1    Succeeded  0         11m
openshift-logging           elasticsearch-rollover-app-1605896100-twvhp    0/1    Failed     0         2d
openshift-logging           elasticsearch-rollover-app-1606130100-hbzg5    0/1    Succeeded  0         11m
openshift-logging           elasticsearch-rollover-audit-1606099500-fd6tb  0/1    Failed     0         8h41m
openshift-logging           elasticsearch-rollover-audit-1606130100-mjk7d  0/1    Succeeded  0         11m
openshift-logging           elasticsearch-rollover-infra-1606041900-2fkcm  0/1    Failed     0         1d
openshift-logging           elasticsearch-rollover-infra-1606130100-dlwsg  0/1    Succeeded  0         11m



elasticsearch-delete-audit-1605896100-54mlp
{"error":{"root_cause":[{"type":"security_exception","reason":"Unexpected exception indices:admin/aliases/get"}],"type":"security_exception","reason":"Unexpected exception indices:admin/aliases/get"},"status":500}
Error while attemping to determine the active write alias: {"error":{"root_cause":[{"type":"security_exception","reason":"Unexpected exception indices:admin/aliases/get"}],"type":"security_exception","reason":"Unexpected exception indices:admin/aliases/get"},"status":500}
elasticsearch-delete-infra-1605896100-nzndt

deleting indices: infra-000070
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "Unexpected exception indices:admin/delete"
      }
    ],
    "type" : "security_exception",
    "reason" : "Unexpected exception indices:admin/delete"
  },
  "status" : 500
}
elasticsearch-rollover-app-1605896100-twvhp
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "Unexpected exception indices:admin/rollover"
      }
    ],
    "type" : "security_exception",
    "reason" : "Unexpected exception indices:admin/rollover"
  },
  "status" : 500
}
elasticsearch-rollover-audit-1606099500-fd6tb
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "Unexpected exception indices:admin/rollover"
      }
    ],
    "type" : "security_exception",
    "reason" : "Unexpected exception indices:admin/rollover"
  },
  "status" : 500
}
elasticsearch-rollover-infra-1606041900-2fkcm
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "Unexpected exception indices:admin/rollover"
      }
    ],
    "type" : "security_exception",
    "reason" : "Unexpected exception indices:admin/rollover"
  },
  "status" : 500
}




VERSIONS:
clusterlogging.4.6.0-202011041933.p0
elasticsearch-operator.4.6.0-202011051050.p0

Comment 1 Jeff Cantrill 2020-11-23 21:35:29 UTC

*** This bug has been marked as a duplicate of bug 1890838 ***


Note You need to log in before you can comment on or make changes to this bug.