Bug 2048793

Summary:	Many Conformance tests are failing in OCP 4.10 with Kuryr
Product:	OpenShift Container Platform	Reporter:	Itay Matza <imatza>
Component:	Networking	Assignee:	Michał Dulko <mdulko>
Networking sub component:	kuryr	QA Contact:	Itay Matza <imatza>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	high	CC:	mdemaced
Version:	4.10	Keywords:	TestBlocker
Target Milestone:	---
Target Release:	4.11.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-08-10 10:45:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2052446

Description Itay Matza 2022-01-31 20:34:18 UTC

Description of problem:
Running kubernetes/conformance in OCP 4.10 with Kuryr ends with many failed tests.
During the test, the Kuryr controller was restarted.


Version-Release number of selected component (if applicable):
OCP 4.10.0-0.nightly-2022-01-29-215708 with Kuryr on top of RHOS-16.2-RHEL-8-20211129.n.1/RHOS-16.1-RHEL-8-20211126.n.1.


How reproducible:
Always.


Steps to Reproduce:
Run kubernetes/conformance from the openshift/origin repo in OCP 4.10 on top of OSP -
 
1. Clone the openshift/origin repo and install dependencies 

2. Build the conformance tests: 
> make WHAT=cmd/openshift-tests

3. Prepare the conformance tests input file by discarding unsupported tests:
> ./openshift-tests run --provider '{"type":"openstack"}' --dry-run kubernetes/conformance |\ngrep -v -e NodePort -e nodePort -e 'session affinity' -e 'NetworkPolicy' -e 'Networking Granular Checks' > conformance-test-results/conformance_tests.txt

4. Run kubernetes/conformance:
> ./openshift-tests run kubernetes/conformance --provider '{"type":"openstack"}' -f conformance-test-results/conformance_tests.txt --output-file conformance-test-results/conformance-test.log --junit-dir=conformance-test-results


Actual results:
Many Conformance tests are failing and the Kuryr controller was restarted.


Expected results:
Conformance tests are passing.


Additional info:
We absorbed that issue is not reproduced when running the conformance test from the origin repo with a checkout previous to https://github.com/openshift/origin/commit/ead65f82a6c92e7609f2c821a05db5d53387d9ed.

Comment 3 Itay Matza 2022-02-15 11:14:16 UTC

Verified on OCP 4.11.0-0.nightly-2022-02-12-075213 on top of OSP RHOS-16.1-RHEL-8-20211126.n.1 with Kuryr.


Verification steps:
1. Clone the openshift/origin repo and install dependencies 

2. Build the conformance tests: 
> make WHAT=cmd/openshift-tests

3. Prepare the conformance tests input file by discarding unsupported tests:
> ./openshift-tests run --provider '{"type":"openstack"}' --dry-run kubernetes/conformance |\ngrep -v -e NodePort -e nodePort -e 'session affinity' -e 'NetworkPolicy' -e 'Networking Granular Checks' > conformance-test-results/conformance_tests.txt

4. Run kubernetes/conformance:
> ./openshift-tests run kubernetes/conformance --provider '{"type":"openstack"}' -f conformance-test-results/conformance_tests.txt --output-file conformance-test-results/conformance-test.log --junit-dir=conformance-test-results


Results:
The conformance test finished with three failed tests out of 332 test cases.

Failed tests:
- no-testclass._sig-cli__Kubectl_client_Guestbook_application_should_create_and_stop_a_working_application___Conformance___Slow___Suite_k8s_
- no-testclass._sig-network__Services_should_serve_a_basic_endpoint_from_pods___Conformance___Suite_openshift/conformance/parallel/minimal___Suite_k8s_
- no-testclass._sig-network__Services_should_serve_multiport_endpoints_from_pods___Conformance___Suite_openshift/conformance/parallel/minimal___Suite_k8s_

Note: The failed tests are known and failing constantly. Look like failures with Service connectivity.

Comment 6 errata-xmlrpc 2022-08-10 10:45:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069