Bug 2052446

Summary: Many Conformance tests are failing in OCP 4.10 with Kuryr
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NetworkingAssignee: MichaƂ Dulko <mdulko>
Networking sub component: kuryr QA Contact: Itay Matza <imatza>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: mdemaced
Version: 4.10Keywords: TestBlocker, Triaged
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-28 12:03:17 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2048793    
Bug Blocks: 2069562    

Comment 5 Itay Matza 2022-03-20 23:42:43 UTC
Verified on OCP 4.10.5 on top of OSP RHOS-16.1-RHEL-8-20211126.n.1 with Kuryr.


Verification steps:
1. Clone the openshift/origin repo and install dependencies 

2. Build the conformance tests: 
> make WHAT=cmd/openshift-tests

3. Prepare the conformance tests input file by discarding unsupported tests:
> ./openshift-tests run --provider '{"type":"openstack"}' --dry-run kubernetes/conformance |\ngrep -v -e NodePort -e nodePort -e 'session affinity' -e 'NetworkPolicy' -e 'Networking Granular Checks' > conformance-test-results/conformance_tests.txt

4. Run kubernetes/conformance:
> ./openshift-tests run kubernetes/conformance --provider '{"type":"openstack"}' -f conformance-test-results/conformance_tests.txt --output-file conformance-test-results/conformance-test.log --junit-dir=conformance-test-results


Results:
The conformance test finished with three failed tests out of 332 test cases.

Failed tests:
- no-testclass._sig-apps__StatefulSet_Basic_StatefulSet_functionality__StatefulSetBasic__Should_recreate_evicted_statefulset__Conformance___Suite_openshift/conformance/parallel/minimal___Suite_k8s_
- no-testclass._sig-cli__Kubectl_client_Guestbook_application_should_create_and_stop_a_working_application___Conformance___Slow___Suite_k8s_
- no-testclass._sig-network__Services_should_serve_multiport_endpoints_from_pods___Conformance___Suite_openshift/conformance/parallel/minimal___Suite_k8s_

Note: Two failed tests are known and failing constantly. Look like failures with Service connectivity.
      The 'apps__StatefulSet_Basic_StatefulSet_functionality__StatefulSetBasic__Should_recreate_evicted_statefulset__Conformance___Suite_openshift' test passed on a rerun.

Comment 9 errata-xmlrpc 2022-03-28 12:03:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.10.6 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1026