Bug 1755000 - kubelet version skew causing CI flakes
Summary: kubelet version skew causing CI flakes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.3.0
Assignee: Hemant Kumar
QA Contact: Liang Xia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-09-24 14:01 UTC by Michal Fojtik
Modified: 2020-05-13 21:25 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-13 21:25:03 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin pull 23855 0 None closed Bug 1755000: Disable tests that need latest kubelet 2020-12-03 11:07:24 UTC
Github openshift origin pull 23861 0 None closed Bug 1755000: Remove whitespace from online expansion test 2020-12-03 11:07:24 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-05-13 21:25:04 UTC

Description Michal Fojtik 2019-09-24 14:01:26 UTC
Description of problem:

[k8s.io] Variable Expansion should allow substituting values in a volume subpath [sig-storage][NodeFeature:VolumeSubpathEnvExpansion] [Suite:openshift/conformance/parallel] [Suite:k8s]
[sig-storage] CSI mock volume CSI Volume expansion should expand volume by restarting pod if attach=off, nodeExpansion=on [Suite:openshift/conformance/parallel] [Suite:k8s]
[sig-storage] CSI mock volume CSI Volume expansion should expand volume by restarting pod if attach=on, nodeExpansion=on [Suite:openshift/conformance/parallel] [Suite:k8s]
[sig-storage] CSI mock volume CSI online volume expansion should expand volume without restarting pod if attach=off, nodeExpansion=on [Suite:openshift/conformance/parallel] [Suite:k8s]
[sig-storage] CSI mock volume CSI online volume expansion should expand volume without restarting pod if attach=on, nodeExpansion=on [Suite:openshift/conformance/parallel] [Suite:k8s]
[sig-storage] CSI mock volume CSI workload information using mock driver contain ephemeral=true when using inline volume [Suite:openshift/conformance/parallel] [Suite:k8s]
[sig-storage] CSI mock volume CSI workload information using mock driver should be passed when podInfoOnMount=true [Suite:openshift/conformance/parallel] [Suite:k8s]
[sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand should resize volume when PVC is edited while pod is using it [Suite:openshift/conformance/parallel] [Suite:k8s]

These are very frequent now because the kubernetes API server and controller manager are on 1.16.0 but the kubelet is still 1.14.x.

These should be resolved automatically when the kubelet is updated.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:

Comment 2 Jan Safranek 2019-09-24 14:08:16 UTC
Full logs: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/openshift_oc/102/pull-ci-openshift-oc-master-e2e-aws/308

It looks like external-attacher expects CSINode is filled:

CSINode ip-10-0-137-224.ec2.internal does not contain driver csi-mock-e2e-csi-mock-volumes-5403`

This will be fixed when RHCOS gets new kubelet, however, we should check that there is no bug in the attacher, it *should* work with older kubelet, so we don't break 3rd party CSI drivers during update.

Comment 9 errata-xmlrpc 2020-05-13 21:25:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.