Description of problem: openshift-apiserver pod has lots of spam logs "E0331...etcdserver: mvcc: required revision has been compacted". There is bug 1537270 found since v3 annoying user. Using this bug for v4 tracker. Not a release blocker but worth addressing in v4. Version-Release number of selected component (if applicable): 4.4.0-0.nightly-2020-03-29-132004 How reproducible: Always Steps to Reproduce: 1. In any normal cluster, check openshift-apiserver logs $ oc logs -n openshift-apiserver apiserver-684c4c58fd-zhrkk Actual results: 1. Lots of below spam logs with the high frequency as shown: ... E0331 07:14:37.992250 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:15:17.454232 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:16:51.789029 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:17:49.103900 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:18:21.208658 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:19:59.232187 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:20:27.581679 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:23:35.898078 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:23:41.608829 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:23:58.527621 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:24:13.190186 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted E0331 07:25:53.514313 1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted ... Expected results: 1. No such spam logs. Or no such high frequency. Additional info:
afaik `mvcc: required revision has been compacted` is a healthy message. Why does it need to be emitted as an error though? The frequency is also concerning.
We agree, it should be fixed, setting the target release to 4.5. The expectation is: - This is a healthy message, so we might need to find a way to make it less frequent. - This should not be labeled as an error message.
Reduced to warning in https://github.com/kubernetes/kubernetes/pull/84602 with 1.18 rebase.
Verified in 4.5.0-0.nightly-2020-05-11-032504. Now OAS log shows it as Warning. $ oc logs -n openshift-apiserver apiserver-79d4f8fd98-9j5bj ... W0511 07:02:16.669394 1 watcher.go:199] watch chan error: etcdserver: mvcc: required revision has been compacted W0511 07:02:16.762114 1 watcher.go:199] watch chan error: etcdserver: mvcc: required revision has been compacted ... W0511 07:11:27.759756 1 watcher.go:199] watch chan error: etcdserver: mvcc: required revision has been compacted W0511 07:11:36.676603 1 watcher.go:199] watch chan error: etcdserver: mvcc: required revision has been compacted ...
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409