Bug 2068600

Summary: cnf-tests: MetalLB: MetalLB deploy should have MetalLB pods in running state. Test is in failed state constantly
Product: OpenShift Container Platform Reporter: Nikita <nkononov>
Component: NetworkingAssignee: Federico Paolinelli <fpaoline>
Networking sub component: Metal LB QA Contact: Nikita <nkononov>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: aos-bugs, cgoncalves, gkopels
Version: 4.9Flags: nkononov: needinfo-
Target Milestone: ---   
Target Release: 4.9.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2094679 (view as bug list) Environment:
Last Closed: 2022-06-30 05:31:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2094679    
Bug Blocks:    

Description Nikita 2022-03-25 18:51:53 UTC
Description of problem:
cnf-tests: MetalLB: MetalLB deploy should have MetalLB pods in running state is failing over last 5 z-stream releases.

• Failure [4.391 seconds]
metallb
/remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:41
  MetalLB deploy
  /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:42
    should have MetalLB pods in running state [It]
    /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:76

    Expected
        <v1.PodPhase>: Pending
    to equal
        <v1.PodPhase>: Running

    /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:95
------------------------------


Version-Release number of selected component (if applicable):
OCP version: Same issue occur on: 4.9.19-4.9.25
cnf-test version: openshift4-cnf-tests:v4.9.7-11

How reproducible:
Run metal-lb test suite on freshly installed OCP 4.9 using openshift4-cnf-tests:v4.9.7-11 container image

Comment 1 Carlos Goncalves 2022-03-29 14:16:34 UTC
Could you please attach/point to relevant logs (e.g. MetalLB Operator logs)?

Comment 6 Carlos Goncalves 2022-06-07 09:10:41 UTC
I cannot reproduce this issue. I deployed OCP 4.9.12 and ran the MetalLB cnf-tests successfully.
Nikita, please provide credentials to a reproducing environment.

Tested container images:
 - registry.redhat.io/openshift4/cnf-tests-rhel8:v4.9.9-9 
 - registry.redhat.io/openshift4/cnf-tests-rhel8:v4.9.8-5 

$ make setup-test-cluster
$ oc get nodes
NAME                                       STATUS   ROLES               AGE   VERSION
ci-ln-1fv6l2t-72292-rhkw5-master-0         Ready    master              26m   v1.22.3+e790d7f
ci-ln-1fv6l2t-72292-rhkw5-master-1         Ready    master              26m   v1.22.3+e790d7f
ci-ln-1fv6l2t-72292-rhkw5-master-2         Ready    master              26m   v1.22.3+e790d7f
ci-ln-1fv6l2t-72292-rhkw5-worker-a-h8pdp   Ready    worker,worker-cnf   16m   v1.22.3+e790d7f
ci-ln-1fv6l2t-72292-rhkw5-worker-b-kjnfg   Ready    worker,worker-cnf   15m   v1.22.3+e790d7f
ci-ln-1fv6l2t-72292-rhkw5-worker-c-n9ghx   Ready    worker              16m   v1.22.3+e790d7f

$ FEATURES="metallb" make feature-deploy
[...]
[INFO] Deploying feature 'metallb' for environment 'deploy'
[INFO] Deployment successful

[cgoncalves@codfish cnf-features-deploy]$ oc get -n metallb-system all
NAME                                                      READY   STATUS    RESTARTS   AGE
pod/metallb-operator-controller-manager-d5b946597-9nnrw   1/1     Running   0          9s

NAME                                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/metallb-operator-controller-manager   1/1     1            1           11s

NAME                                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/metallb-operator-controller-manager-d5b946597   1         1         1       11s


$ podman run --rm -v /home/cgoncalves/Downloads:/kubeconfig:z -e KUBECONFIG=/kubeconfig/kubeconfig -e IS_OPENSHIFT=true registry.redhat.io/openshift4/cnf-tests-rhel8:v4.9.9-9 /usr/bin/test-run.sh  -ginkgo.focus="metallb"
[...]
• [SLOW TEST:30.558 seconds]
metallb
/remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:41
  MetalLB deploy
  /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:42
    should have MetalLB pods in running state
    /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:76
------------------------------
•••
[...]

Comment 7 Greg Kopels 2022-06-20 12:30:43 UTC
On the last two 4.9 z-streams this test case has passed:

14:12:24  • [SLOW TEST:41.251 seconds]
14:12:24  metallb
14:12:24  /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:41
14:12:24    MetalLB deploy
14:12:24    /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:42
14:12:24      should have MetalLB pods in running state
14:12:24      /remote-source/app/vendor/github.com/metallb/metallb-operator/test/e2e/functional/tests/e2e.go:76
14:12:24  ------------------------------
14:12:35  •••

Comment 10 errata-xmlrpc 2022-06-30 05:31:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.9.40 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:5180