Bug 1371426 - Log throttling settings did not effect to .operations index
Summary: Log throttling settings did not effect to .operations index
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.3.0
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Luke Meyer
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-30 08:11 UTC by Xia Zhao
Modified: 2017-03-08 18:43 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-18 12:52:47 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0066 0 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.4 RPM Release Advisory 2017-01-18 17:23:26 UTC

Comment 1 Luke Meyer 2016-08-30 13:11:12 UTC
This may be a side effect of how .operations is combined from multiple sources (none of which are named ".operations"). @Rich any thoughts?

Comment 2 Rich Megginson 2016-08-30 14:17:37 UTC
oc exec $fluentdpod -- ls -alrtFR /etc/fluent

Comment 5 Rich Megginson 2016-11-08 21:51:32 UTC
oc exec $fluentdpod -- ls -alrtFR /etc/fluent

Comment 6 Junqi Zhao 2016-12-05 09:46:50 UTC
Verified according to xiazhao's steps with logging 3.4.0, Log throttling settings take effect on .operation index.Please set this issue to ON_QA.
Image ID:
openshift3/logging-fluentd    7b11a29c82c1
openshift3/logging-elasticsearch    6716a0ad8b2b
openshift3/logging-auth-proxy    ec334b0c2669
openshift3/logging-kibana    7fc9916eea4d
openshift3/logging-curator    9af78fc06248

.operation
before throttling : The log entries arrived per 1, 3 ,5 minutes are: 1478, 4725, 8748

after throttling : The log entries arrived per 1, 3 ,5 minutes are: 1039, 2904, 5202

oc exec $fluentdpod -- ls -alrtFR /etc/fluent info:
# oc exec logging-fluentd-1rubw -- ls -alrtFR /etc/fluent
/etc/fluent:
total 8
drwxr-xr-x.  2 root root   41 Nov 18 23:15 plugin/
lrwxrwxrwx.  1 root root   38 Nov 18 23:15 fluent.conf -> /etc/fluent/configs.d/user/fluent.conf
drwxr-xr-x.  5 root root   50 Nov 18 23:15 configs.d/
drwxrwxrwt.  3 root root  140 Dec  5 04:05 keys/
drwxr-xr-x. 52 root root 4096 Dec  5 04:05 ../
drwxr-xr-x.  5 root root   68 Dec  5 04:05 ./

/etc/fluent/plugin:
total 4
-rw-r--r--. 1 root root 4091 Nov 18 23:14 filter-common-data-model.rb
drwxr-xr-x. 2 root root   41 Nov 18 23:15 ./
drwxr-xr-x. 5 root root   68 Dec  5 04:05 ../

/etc/fluent/configs.d:
total 4
drwxr-xr-x. 2 root root 4096 Nov 18 23:14 openshift/
drwxr-xr-x. 5 root root   50 Nov 18 23:15 ./
drwxrwxrwx. 3 root root  135 Dec  5 04:05 user/
drwxr-xr-x. 5 root root   68 Dec  5 04:05 ../
drwxrwxrwx. 2 root root  144 Dec  5 04:05 dynamic/

/etc/fluent/configs.d/openshift:
total 72
-rw-r--r--. 1 root root   36 Nov 18 23:14 system.conf
-rw-r--r--. 1 root root  281 Nov 18 23:14 output-operations.conf
-rw-r--r--. 1 root root  610 Nov 18 23:14 output-es-ops-config.conf
-rw-r--r--. 1 root root  601 Nov 18 23:14 output-es-config.conf
-rw-r--r--. 1 root root  191 Nov 18 23:14 output-applications.conf
-rw-r--r--. 1 root root 5381 Nov 18 23:14 filter-syslog-record-transform.conf
-rw-r--r--. 1 root root 4578 Nov 18 23:14 filter-retag-journal.conf
-rw-r--r--. 1 root root  217 Nov 18 23:14 filter-kibana-transform.conf
-rw-r--r--. 1 root root 2206 Nov 18 23:14 filter-k8s-record-transform.conf
-rw-r--r--. 1 root root  314 Nov 18 23:14 filter-k8s-meta.conf
-rw-r--r--. 1 root root   64 Nov 18 23:14 filter-exclude-journal-debug.conf
-rw-r--r--. 1 root root  604 Nov 18 23:14 filter-common-data-model.conf
-rw-r--r--. 1 root root  700 Nov 18 23:14 es-ops-copy-config.conf
-rw-r--r--. 1 root root  688 Nov 18 23:14 es-copy-config.conf
-rw-r--r--. 1 root root 4061 Nov 18 23:14 README
drwxr-xr-x. 2 root root 4096 Nov 18 23:14 ./
drwxr-xr-x. 5 root root   50 Nov 18 23:15 ../

/etc/fluent/configs.d/user:
total 0
drwxr-xr-x. 5 root root  50 Nov 18 23:15 ../
lrwxrwxrwx. 1 root root  27 Dec  5 04:05 throttle-config.yaml -> ..data/throttle-config.yaml
lrwxrwxrwx. 1 root root  26 Dec  5 04:05 secure-forward.conf -> ..data/secure-forward.conf
lrwxrwxrwx. 1 root root  18 Dec  5 04:05 fluent.conf -> ..data/fluent.conf
lrwxrwxrwx. 1 root root  33 Dec  5 04:05 ..data -> ..129812_05_12_04_05_08.759086152/
drwxr-xr-x. 2 root root  80 Dec  5 04:05 ..129812_05_12_04_05_08.759086152/
drwxrwxrwx. 3 root root 135 Dec  5 04:05 ./

/etc/fluent/configs.d/user/..129812_05_12_04_05_08.759086152:
total 12
-rw-r--r--. 1 root root  200 Dec  5 04:05 throttle-config.yaml
-rw-r--r--. 1 root root  563 Dec  5 04:05 secure-forward.conf
-rw-r--r--. 1 root root 1243 Dec  5 04:05 fluent.conf
drwxrwxrwx. 3 root root  135 Dec  5 04:05 ../
drwxr-xr-x. 2 root root   80 Dec  5 04:05 ./

/etc/fluent/configs.d/dynamic:
total 16
drwxr-xr-x. 5 root root  50 Nov 18 23:15 ../
-rw-r--r--. 1 root root 548 Dec  5 04:05 input-syslog-default-syslog.conf
-rw-r--r--. 1 root root 251 Dec  5 04:05 input-docker-default-docker.conf
-rw-r--r--. 1 root root   1 Dec  5 04:05 es-ops-copy-config.conf
-rw-r--r--. 1 root root   1 Dec  5 04:05 es-copy-config.conf
drwxrwxrwx. 2 root root 144 Dec  5 04:05 ./

/etc/fluent/keys:
total 0
lrwxrwxrwx. 1 root root  10 Dec  5 04:05 key -> ..data/key
lrwxrwxrwx. 1 root root  11 Dec  5 04:05 cert -> ..data/cert
lrwxrwxrwx. 1 root root   9 Dec  5 04:05 ca -> ..data/ca
lrwxrwxrwx. 1 root root  33 Dec  5 04:05 ..data -> ..129812_05_12_04_05_08.311122183/
drwxr-xr-x. 2 root root 100 Dec  5 04:05 ..129812_05_12_04_05_08.311122183/
drwxrwxrwt. 3 root root 140 Dec  5 04:05 ./
drwxr-xr-x. 5 root root  68 Dec  5 04:05 ../

/etc/fluent/keys/..129812_05_12_04_05_08.311122183:
total 12
-rw-r--r--. 1 root root 1704 Dec  5 04:05 key
-rw-r--r--. 1 root root 1208 Dec  5 04:05 cert
-rw-r--r--. 1 root root 1078 Dec  5 04:05 ca
drwxrwxrwt. 3 root root  140 Dec  5 04:05 ../
drwxr-xr-x. 2 root root  100 Dec  5 04:05 ./

Comment 7 Xia Zhao 2016-12-06 00:58:52 UTC
Verified on 3.4.0 according to comment #6

Comment 9 errata-xmlrpc 2017-01-18 12:52:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0066


Note You need to log in before you can comment on or make changes to this bug.