periodic-ci-openshift-release-master-ci-4.9-e2e-openstack-parallel is failing frequently in CI, see: https://sippy.ci.openshift.org/sippy-ng/jobs/4.9/analysis?filters=%7B%22items%22%3A%5B%7B%22columnField%22%3A%22name%22%2C%22operatorValue%22%3A%22equals%22%2C%22value%22%3A%22periodic-ci-openshift-release-master-ci-4.9-e2e-openstack-parallel%22%7D%5D%7D&tests=%5Bsig-network%5D%20Networking%20IPerf2%20%5BFeature%3ANetworking-Performance%5D%20should%20run%20iperf2%20%5BSuite%3Aopenshift%2Fconformance%2Fparallel%5D%20%5BSuite%3Ak8s%5D and https://testgrid.k8s.io/redhat-openshift-ocp-release-4.9-informing#periodic-ci-openshift-release-master-ci-4.9-e2e-openstack-parallel Example job: https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.9-e2e-openstack-parallel/1433006778466963456 [sig-network] Networking IPerf2 [Feature:Networking-Performance] should run iperf2 [Suite:openshift/conformance/parallel] [Suite:k8s] expand_less fail [github.com/onsi/ginkgo.0-origin.0+incompatible/internal/leafnodes/runner.go:113]: Sep 1 10:54:07.533: iperf2 MB/s received below baseline of 90 for client hdqi2rrj-96d0b-2zm5q-worker-0-kl4r9 to server hdqi2rrj-96d0b-2zm5q-worker-0-b52nq: 59
It seems like these jobs would have otherwise passed if the Iperf2 job was not running. We don't care to test the networking performance of our CI hardware anyway, so we are moving to skip this test.
post: https://github.com/kubernetes/kubernetes/pull/104845
Backport to kubernetes 1.22
https://github.com/kubernetes/kubernetes/pull/104859
The upstream fix has been backported to 1.22, but has not yet been included in a tagged release. The most recent tag was 1.22.1, so we can expect the fix to be included in 1.22.2. I believe the path to resolving this bug is: 1. Commit a0bebe70ef268e097d775a8e76a725e81a548fad is included in upstream tag 1.22.2 2. openshift/kubernetes:master rebases on upstream tag 1.22.2 3. openshift/origin revendors k8s.io/kubernetes on openshift/kubernetes:master Ideally all of these things should happen automatically over the next few weeks.
*** Bug 2006346 has been marked as a duplicate of this bug. ***
> Ideally all of these things should happen automatically over the next few weeks. This is failing quite a bit, not on just the openstack job -- can we bring the fix from upstream directly? https://github.com/openshift/kubernetes/pull/980
Even though the openshift/kubernetes change merged 2 days ago, we're still seeing issues in our CI: https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.10-e2e-openstack-parallel/1443518614392016896 I assume it needs to be re-vendored in openshift/origin now?
Looks fine, I am not seeing new runs failed because of iPerf2 for now.
Moving back to ON_QA as we need to move the bug to VERIFIED in order for the backport to be valid.
Self-verifying. This iperf2 test is no longer an issue on 4.10 as shown by: https://search.ci.openshift.org/?search=should+run+iperf2&maxAge=48h&context=1&type=bug%2Bjunit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
Removing the Triaged keyword because: * the QE automation assessment (flag qe_test_coverage) is missing
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056