Description of problem: log deletion only effect to one index (the first one specified in curator-config.yaml) among those with duplicated policies. Similar old bug: https://bugzilla.redhat.com/show_bug.cgi?id=1352486 Version-Release number of selected component (if applicable): Image: logging-curator v3.6 52f85e17fd38 6 hours ago 221.5 MB # openshift version openshift v3.6.136 kubernetes v1.6.1+5115d708d7 etcd 3.2.1 How reproducible: always Steps to Reproduce: 1. Note down the current indices for your interest in openshift: $ oc exec $curator-pod -- curator --host logging-es --use_ssl --certificate /etc/curator/keys/ca --client-cert /etc/curator/keys/cert --client-key /etc/curator/keys/key --loglevel ERROR show indices --all-indices project.day1-dup.2646499a-ca50-457d-9cc4-5179bda279c7.2016.2016.09.01 project.day1-dup.4f47d263-c25e-4def-99cb-53963edab8a4.2016.2017.7.7 project.day1-tri.4dfc53c1-9176-4d89-be0b-f2b701a3eee2.2016.2017.7.7 project.day1-tri.62edf619-4948-4cbf-bbf4-c69db628832d.2016.2016.08.29 project.day1.3464b2d1-f4c6-4900-813a-8c17fa0914e5.2016.2017.7.7 project.day1.fd26339e-87fc-49bd-bc19-860d43eac070.2016.2016.08.29 2. Configure curator settings like this (by editting the configmap) : $ oc edit configmap logging-curator -o yaml ... day1: delete: days: 1 day-dup: delete: days: 1 day-tri: delete: days: 1 ... 3. Scale down curator pod: oc scale rc logging-curator-1 --replicas=0 4. Scale up curator pod: oc scale rc logging-curator-1 --replicas=1 5. Check the index deletion result: $ oc exec $curator-pod -- curator --host logging-es --use_ssl --certificate /etc/curator/keys/ca --client-cert /etc/curator/keys/cert --client-key /etc/curator/keys/key --loglevel ERROR show indices --all-indices project.day1-dup.2646499a-ca50-457d-9cc4-5179bda279c7.2016.2016.09.01 project.day1-dup.4f47d263-c25e-4def-99cb-53963edab8a4.2016.2017.7.7 project.day1-tri.4dfc53c1-9176-4d89-be0b-f2b701a3eee2.2016.2017.7.7 project.day1-tri.62edf619-4948-4cbf-bbf4-c69db628832d.2016.2016.08.29 project.day1.3464b2d1-f4c6-4900-813a-8c17fa0914e5.2016.2017.7.7 Actual results: old logs more than 1 day for project day-dup and day-tri are not deleted, only those for project day1 deleted Expected results: old logs more than 1 day for project day-dup and day-tri should also be deleted Additional info:
The indices all start with the prefix "project.day1-" but your curator config project names _do not_: day-dup: delete: days: 1 day-tri: delete: days: 1 It should work if you either change your index names to be "project.day-dup" and "project.day-tri", or change your curator config like this: day1-dup: delete: days: 1 day1-tri: delete: days: 1 In addition - curator works with the index naming scheme used by fluentd/elasticsearch: project.name.uuid.YYYY.MM.DD Where MM and DD are the _2_ digit month and day e.g. 07 instead of 7. Using any other index naming convention may work but is not supported.
(In reply to Rich Megginson from comment #1) > The indices all start with the prefix "project.day1-" but your curator > config project names _do not_: Woo, my appologize for the mistake, thanks for pointing it out, Rich! > Where MM and DD are the _2_ digit month and day e.g. 07 instead of 7. Using > any other index naming convention may work but is not supported. Thanks for the info, I'll modify my script to make the timestamps in _2_ digit format when addming messages into curator.