Bug 1820468 - Lots of log spam 'loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist" annoying
Summary: Lots of log spam 'loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed wi...
Keywords:
Status: CLOSED DUPLICATE of bug 1819053
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.4
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.4.z
Assignee: Sergiusz Urbaniak
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On: 1819053
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-03 07:07 UTC by Xingxing Xia
Modified: 2020-04-20 07:50 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1819053
Environment:
Last Closed: 2020-04-20 07:50:24 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Xingxing Xia 2020-04-03 07:07:04 UTC
As shown in the comment, 4.4 also reproduces. The log spam is so much, seen in daily build test, that I think it is worthy to address in 4.4 too, not must within 4.4.0, even to be addressed in 4.4.z is very expected. Thus cloning a 4.4.z one.

+++ This bug was initially created as a clone of Bug #1819053 +++

Clusters deployed with the 1.18 origin rebase PR [1] have the following in the apiserver logs every 60s:

controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.

This problem doesn't appear to be limited to the rebase PR, as per a longstanding github issue [2].

Beyond eliminating the log spew, it would be helpful to know what functional impact - if any - these logged errors represent.

1: https://github.com/openshift/origin/pull/24719
2: https://github.com/coreos/kube-prometheus/issues/304

--- Additional comment from Xingxing Xia on 2020-03-31 17:08:18 CST ---

4.4.0-0.nightly-2020-03-29-132004 also reproduces:
$ oc logs -n openshift-kube-apiserver kube-apiserver-ip-10-0-134-38.ap-northeast-2.compute.internal -c kube-apiserver
...
I0331 02:35:46.842201       1 aggregator.go:226] Updating OpenAPI spec because v1.route.openshift.io is updated
I0331 02:35:47.317974       1 aggregator.go:229] Finished OpenAPI spec generation after 475.760307ms
I0331 02:35:47.318047       1 aggregator.go:226] Updating OpenAPI spec because v1beta1.metrics.k8s.io is updated
I0331 02:35:47.818875       1 aggregator.go:229] Finished OpenAPI spec generation after 500.807236ms
I0331 02:35:47.818956       1 aggregator.go:226] Updating OpenAPI spec because v1.user.openshift.io is updated
I0331 02:35:48.273538       1 aggregator.go:229] Finished OpenAPI spec generation after 454.560076ms
...snipped...
I0331 02:35:59.218097       1 aggregator.go:226] Updating OpenAPI spec because v1.route.openshift.io is updated
I0331 02:36:00.378834       1 aggregator.go:229] Finished OpenAPI spec generation after 1.160710014s
E0331 02:36:00.382542       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
I0331 02:36:00.382560       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0331 02:36:00.649455       1 aggregator.go:226] Updating OpenAPI spec because v1.user.openshift.io is updated
...snipped...
I0331 02:36:08.140496       1 aggregator.go:226] Updating OpenAPI spec because v1.template.openshift.io is updated                     
I0331 02:36:09.401918       1 aggregator.go:229] Finished OpenAPI spec generation after 1.261388097s                                   
E0331 02:37:00.386352       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
I0331 02:37:00.386371       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0331 02:39:00.397969       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
I0331 02:39:00.397986       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0331 02:40:44.528108       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
I0331 02:40:44.528125       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0331 02:41:44.536408       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
I0331 02:41:44.536428       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0331 02:43:44.543085       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exist
...all following logs are only above spams now...

--- Additional comment from Xingxing Xia on 2020-03-31 17:12:34 CST ---

Like bug 1537270#c33 / bug 1819103 log spams annoying customer, this bug also annoys I think. Thus updating bug field.


Note You need to log in before you can comment on or make changes to this bug.