Bug 1754788 - Master-controllers log spammed with "The resourceVersion for the provided watch is too old." warning messages"
Summary: Master-controllers log spammed with "The resourceVersion for the provided wat...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-controller-manager
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 3.11.z
Assignee: Sally
QA Contact: zhou ying
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-24 05:44 UTC by K Chandra Sekar
Modified: 2020-02-24 08:20 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-18 01:34:37 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:3139 None None None 2019-10-18 01:34:50 UTC

Description K Chandra Sekar 2019-09-24 05:44:59 UTC
Description of problem:

master-controllers log spammed with "The resourceVersion for the provided watch is too old." warning messages. 

This warning messages are converted to Information messages on OCP v3.9 with the errata[1] created in the bugzilla[2] but we can see still see these messages are listed as Warning and not as Information in OCP v3.11 

~~~~~
oc logs pod/master-controllers-xxx

W0919 11:23:34.983377       1 reflector.go:272] k8s.io/client-go/informers/factory.go:130: watch of *v1.PersistentVolumeClaim ended with: The resourceVersion for the provided watch is too old.
W0919 11:23:48.902273       1 reflector.go:272] k8s.io/client-go/informers/factory.go:130: watch of *v1.StorageClass ended with: The resourceVersion for the provided watch is too old.
W0919 11:30:16.382786       1 reflector.go:272] k8s.io/client-go/informers/factory.go:130: watch of *v1beta1.StatefulSet ended with: The resourceVersion for the provided watch is too old.
W0919 11:32:04.568718       1 reflector.go:272] k8s.io/client-go/informers/factory.go:130: watch of *v1beta1.ReplicaSet ended with: The resourceVersion for the provided watch is too old.

~~~~~

[1] - https://access.redhat.com/errata/RHBA-2019:2550

[2] - https://bugzilla.redhat.com/show_bug.cgi?id=1573460


Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
Check the controller pod logs in kube-system namespace to witness these logs

Actual results: The message "The resourceVersion for the provided watch is too old" is listed as a Warning Message.


Expected results: The message "The resourceVersion for the provided watch is too old" should be a Informational message not a Warning Message.


Additional info:

Comment 1 Maciej Szulik 2019-09-24 08:16:04 UTC
Sally you were working on https://bugzilla.redhat.com/show_bug.cgi?id=1573460 and picked https://github.com/kubernetes/kubernetes/pull/78991 into 3.9 in https://github.com/openshift/origin/pull/23284,
mind re-picking this into origin 3.11

Comment 5 zhou ying 2019-10-12 08:29:24 UTC
Confirmed with version, the issue has fixed:
[root@ip-172-18-10-58 ~]# oc version
oc v3.11.153
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ip-172-18-10-58.ec2.internal:8443
openshift v3.11.153
kubernetes v1.11.0+d4cacc0


[root@ip-172-18-10-58 ~]# oc logs -f master-controllers-xxxxx |grep "The resourceVersion for the provided watch"
I1012 04:10:49.317530       1 reflector.go:274] k8s.io/client-go/informers/factory.go:130: watch of *v1.ResourceQuota ended with: The resourceVersion for the provided watch is too old.
I1012 04:11:29.481605       1 reflector.go:274] k8s.io/client-go/informers/factory.go:130: watch of *v1beta1.Ingress ended with: The resourceVersion for the provided watch is too old.
I1012 04:11:35.296693       1 reflector.go:274] github.com/openshift/client-go/build/informers/externalversions/factory.go:101: watch of *v1.Build ended with: The resourceVersion for the provided watch is too old.
I1012 04:11:41.212701       1 reflector.go:274] k8s.io/client-go/informers/factory.go:130: watch of *v1beta1.CronJob ended with: The resourceVersion for the provided watch is too old.
I1012 04:12:03.398060       1 reflector.go:274] k8s.io/client-go/informers/factory.go:130: watch of *v1beta1.PodSecurityPolicy ended with: The resourceVersion for the provided watch is too old.
I1012 04:12:18.470360       1 reflector.go:274] k8s.io/client-go/informers/factory.go:130: watch of *v1beta1.VolumeAttachment ended with: The resourceVersion for the provided watch is too old.
I1012 04:12:23.294926       1 reflector.go:274] k8s.io/client-go/informers/factory.go:130: watch of *v1.HorizontalPodAutoscaler ended with: The resourceVersion for the provided watch is too old.
I1012 04:12:33.033484       1 reflector.go:274] k8s.io/client-go/informers/factory.go:130: watch of *v1beta1.PodDisruptionBudget ended with: The resourceVersion for the provided watch is too old.
I1012 04:12:34.241481       1 reflector.go:274] github.com/openshift/origin/pkg/quota/generated/informers/internalversion/factory.go:101: watch of *quota.ClusterResourceQuota ended with: The resourceVersion for the provided watch is too old.

Comment 7 errata-xmlrpc 2019-10-18 01:34:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3139


Note You need to log in before you can comment on or make changes to this bug.