We want to change the third-party network plugin certification instructions to use openshift-tests rather than the upstream e2e.test. To verify: build origin from source or pull the "openshift-tests" image and run openshift-tests run openshift/network/third-party and it should run about 60 tests, some of which are expected to fail under openshift-sdn. (I think if you ran them under ovn-kubernetes they'd all pass though... But anyway, it doesn't matter whether the tests pass or not, just that (a) it runs tests rather than saying "no such test suite", and (b) the number of passed+failed tests selected by this suite is somewhere around 60 right now.)
Run test on 4.7.0-0.nightly-2021-01-26-044139 with build origin from source on OVN cluster. test result is about 53 cases, and 4 cases failed. [sig-arch] Monitor cluster while tests execute Failing tests: [sig-network] Services should be able to change the type from ClusterIP to ExternalName [Conformance] [Suite:openshift/conformance/parallel/minimal] [Suite:k8s] [sig-network] Services should be able to change the type from ExternalName to ClusterIP [Conformance] [Suite:openshift/conformance/parallel/minimal] [Suite:k8s] [sig-network] Services should be able to change the type from ExternalName to NodePort [Conformance] [Suite:openshift/conformance/parallel/minimal] [Suite:k8s] [sig-network] Services should be able to change the type from NodePort to ExternalName [Conformance] [Suite:openshift/conformance/parallel/minimal] [Suite:k8s] [sig-network] Services should be able to create a functioning NodePort service [Conformance] [Suite:openshift/conformance/parallel/minimal] [Suite:k8s] error: 5 fail, 48 pass, 0 skip (8m42s)
Weird... I just ran on AWS with ovn-kubernetes, 4.7.0-0.nightly-2021-01-26-082025, and got 53 pass, 0 skip (8m7s) So anyway, not sure why you got 5 failures (something is flaking maybe?) but it looks like the test suite is correct
Aha, I ran again on one US client, 53 pass. but 5 failed in Beijing client. So this should be Beijing network issue Flaky invariants: [sig-arch] Monitor cluster while tests execute 53 pass, 0 skip (7m26s) Move this bug to verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633