Bug 1517875
| Summary: | RFE: Allow regular expressions in curator settings | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Bahaddin <bcankurt> | |
| Component: | RFE | Assignee: | Josef Karasek <jkarasek> | |
| Status: | CLOSED ERRATA | QA Contact: | Anping Li <anli> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 3.5.1 | CC: | aos-bugs, dzhukous, jcantril, jkarasek, jokerman, mmccomas, rmeggins, sradco, tkatarki, trankin, wsun | |
| Target Milestone: | --- | |||
| Target Release: | 3.9.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Enhancement | ||
| Doc Text: |
Feature: Allow regular expressions in curator settings
Reason: Customer request
Result: A special tag can be used in curator settings to specify custom regular expressions.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1541948 (view as bug list) | Environment: | ||
| Last Closed: | 2018-03-28 14:13:03 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1541948, 1541952 | |||
ovirt/rhv would like this feature as well Bahaddin, we're currently moving to curator 5.2, which allows regexes in its configuration. See https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/filtertype_pattern.html#_regex Let me know if that helps you. Thanks (In reply to Josef Karasek from comment #2) > Bahaddin, we're currently moving to curator 5.2, which allows regexes in its > configuration. See > https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/ > filtertype_pattern.html#_regex > > Let me know if that helps you. Thanks Josef, is there anything we can do in current version of curator? (In reply to Josef Karasek from comment #2) > Bahaddin, we're currently moving to curator 5.2, which allows regexes in its > configuration. See > https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/ > filtertype_pattern.html#_regex > > Let me know if that helps you. Thanks Hi Josef, which OCP version will be using the curator 5.2? Thanks, Bahaddin We're hoping to land this upgrade in early 2018. I understand that this might not satisfied your needs. I'll look into extending the current config file format to accept regular expressions as well. The latest image logging-curator/images/v3.9.0-0.53.0.0 won't include the fix. Waiting the new image. @josef, The curator report error, Could you Check if the configmap are correct?
[root@ip-172-18-5-104 bin]# oc get configmap logging-curator -o yaml
apiVersion: v1
data:
config.yaml: |
myapp-dev:
delete:
days: 1
myapp-qe:
delete:
weeks: 1
.operations:
delete:
weeks: 2
.defaults:
delete:
days: 31
runhour: 0
runminute: 0
timezone: America/New_York
.regex:
- pattern: '^project\..+\-dev\..*$'
delete:
days: 15
- pattern: '^project\..+\-test\..*$'
delete:
days: 30
kind: ConfigMap
[root@ip-172-18-5-104 bin]# oc logs logging-curator-1-6sptm
The project name must match this regex: [^[a-z0-9]([-a-z0-9]*[a-z0-9])?$] This does not match: [.regex]
The test fails because none of the latest builds have the latest code. I'm looking into it.
The indentation of the config should be:
config.yaml: |
myapp-dev:
delete:
days: 1
myapp-qe:
delete:
weeks: 1
.operations:
delete:
weeks: 2
.defaults:
delete:
days: 31
runhour: 0
runminute: 0
timezone: America/New_York
.regex:
- pattern: '^project\..+\-dev\..*$'
delete:
days: 15
- pattern: '^project\..+\-test\..*$'
delete:
days: 30
You can find more info here https://github.com/openshift/origin-aggregated-logging/blob/release-3.9/curator/README.md
Moving back to RFE as this was a change enhancement. Updated dist-git repos to include latest files due to bad merge. Requires new build to validate. The curator works as expected with openshift3/logging-curator/images/v3.9.11-1 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0489 |
Description of problem: I would like to delete some indices according to the regular expressions I provide, so that I am not forced to enter individual project names into the configs. Right now I am creating a setting yaml with a one-liner and add it to the configmap, manually. But whenever new projects are added to the cluster I will have to maintain the list. Version-Release number of selected component (if applicable): 3.5.1 How reproducible: Steps to Reproduce: 1. Install Cluster 2. Deploy logging with the provided playbook 3. Add the following to the configmap logging-curator .+nonprod.+: delete: days: 5 In order to delete the indices that belong to *nonprod projects. Actual results: Regex characters are escaped, as a result the indices are not deleted.