Bug 1875272

Summary: Curator failed with "Failed to complete action: delete_indices"
Product: OpenShift Container Platform Reporter: Mohamed Tleilia <mtleilia>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED WONTFIX QA Contact: Anping Li <anli>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.11.0CC: aos-bugs
Target Milestone: ---   
Target Release: 3.11.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-01 18:43:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 1 Jeff Cantrill 2020-09-12 01:07:43 UTC
Not to be addressed UpcomingSprint

Comment 2 Jeff Cantrill 2020-10-01 18:43:50 UTC
(In reply to Mohamed Tleilia from comment #0)
> Description of problem:
> Curator pods failed when trying to clean up indexes:
> ~~~
> # oc logs logging-curator-1598500800-vhc98
> 2020-08-27 04:00:16,442 INFO    Found curator configuration in
> [/etc/curator/settings/config.yaml]
> 2020-08-27 04:00:16,445 INFO    Converting config file.
> 2020-08-27 04:01:32,487 ERROR     The following indices failed to delete on
> try #1:
> 2020-08-27 04:01:32,487 ERROR    
> ---project.2t61-p.c2ccc1e5-525f-11ea-8550-b88303501cee.2020.08.12
> 2020-08-27 04:01:32,487 ERROR    
> ---project.qa10-d.42163971-f60f-11e7-80ba-68b5996b6fee.2020.08.12
> 2020-08-27 04:01:32,490 ERROR     Failed to complete action: delete_indices.
> <class 'curator.exceptions.FailedExecution'>: Exception encountered.  Rerun
> with loglevel DEBUG and/or check Elasticsearch logs for more information.

This looks like there is potentially an issue between the time it evaluates the list of potential issues and the time it removes them.

> How reproducible:
> Happened from time to time. The client used to delete the Errored Curator
> Pod, often Curator job succeeded on next schedule. 

Given is intermittent and the job succeeds on subsequent runs with presumably the indices being removed, CLOSING WONTFIX.  Curator is deprecated in 4.x in favor of the new index management.

Please reopen if you find the older indices are still hanging around after successful job executions