Bug 1908363
| Summary: | Applying NodeNetworkConfigurationPolicy for different NIC than default disables br-ex bridge and nodes lose connectivity | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | Marius Cornea <mcornea> | ||||||||||||||
| Component: | Networking | Assignee: | Petr Horáček <phoracek> | ||||||||||||||
| Status: | CLOSED ERRATA | QA Contact: | Meni Yakove <myakove> | ||||||||||||||
| Severity: | urgent | Docs Contact: | |||||||||||||||
| Priority: | unspecified | ||||||||||||||||
| Version: | 2.6.0 | CC: | cnv-qe-bugs, ysegev | ||||||||||||||
| Target Milestone: | --- | ||||||||||||||||
| Target Release: | 2.6.0 | ||||||||||||||||
| Hardware: | Unspecified | ||||||||||||||||
| OS: | Unspecified | ||||||||||||||||
| Whiteboard: | |||||||||||||||||
| Fixed In Version: | kubernetes-nmstate-handler-container-v2.6.0-8 | Doc Type: | If docs needed, set a value | ||||||||||||||
| Doc Text: | Story Points: | --- | |||||||||||||||
| Clone Of: | Environment: | ||||||||||||||||
| Last Closed: | 2021-03-10 11:22:41 UTC | Type: | Bug | ||||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||||
| Documentation: | --- | CRM: | |||||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||||
| Embargoed: | |||||||||||||||||
| Bug Depends On: | 1904889 | ||||||||||||||||
| Bug Blocks: | |||||||||||||||||
| Attachments: |
|
||||||||||||||||
Created attachment 1739654 [details]
nmcli con output before applying NNCP
Created attachment 1739655 [details]
nmcli con output after applying NNCP
Created attachment 1739656 [details]
ip a output before applying NNCP
Created attachment 1739657 [details]
ip a output after applying NNCP
Created attachment 1739658 [details]
nmstate pod log from one of the affected nodes
Thanks for reporting this, Marius. I believe this is caused by https://bugzilla.redhat.com/show_bug.cgi?id=1903712. The build of knmstate fixing this is on its way to 2.6. It will be available as soon as it finishes its run through our D/S pipeline. Verified on OCP 4.7.0 CNV 2.6.0 nmstate-0.3.4-17.el8_3.noarch kubernetes-nmstate-handler-rhel8@sha256:d7e6493bde74eaa72748086cf6a2abaa7abefdc1d78cb273df763a910dfb4d9 I applied the NodeNetworkConfigurationPolicy from the bug description, with changing the interface to a secondary interface available on my nodes (ens8). Result: 1. NNCP successfully configured. 2. br-ex interface found in "ip a" output on one of the worker nodes (in the output provided in the bug original description - it was missing). 3. In the "nmcli c show" output of one of the worker nodes: ovs-if-br-ex appears as ovs-interface of br-ex ovs-port-br-ex appears as ovs-port of br-ex (in the output provided in the bug original description they were missing). In addition - the new VLAN interfaces also appear. "nmcli c show" output (after applying the policy) is given below: NAME UUID TYPE DEVICE ovs-if-br-ex caee1e5d-8045-4523-a85f-4ec1af0b8a28 ovs-interface br-ex bigip-ha 2bf56523-aa17-42aa-b072-cfa2519b171f bridge bigip-ha bigip-mgmt 430eeca8-b54a-4f3e-a135-47e19a61cfea bridge bigip-mgmt br-ex db52a338-97eb-41bc-a59a-2a022e32abba ovs-bridge br-ex ens8.375 23ad82cc-5645-4e7e-b9fe-0b10aae334d0 vlan ens8.375 ens8.376 3d434b47-4842-4e11-a36d-f101ae38b7b6 vlan ens8.376 ovs-if-phys0 8e7ebff2-4766-4a71-a630-f6aebaf1949a ethernet ens3 ovs-port-br-ex 28c79aac-ae4e-4ab1-89ed-25ac1b28ad45 ovs-port br-ex ovs-port-phys0 5a8d5ee6-bc63-4933-a019-8bed626046fc ovs-port ens3 An additional note - the bug was verified on an OVN-Kubernetes cluster (the br-ex OVS bridge is not deployed on an OpenshiftSDN cluster). Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Virtualization 2.6.0 security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:0799 |
Created attachment 1739653 [details] NetworkManager journal Description of problem: Applying NodeNetworkConfigurationPolicy for different NIC than default disables br-ex bridge and nodes lose connectivity. Version-Release number of selected component (if applicable): 2.6.0 iib registry-proxy.engineering.redhat.com/rh-osbs/iib:31420 How reproducible: 100% Steps to Reproduce: 1. Deploy OCP 4.7 via baremetal IPI flow 2. Deploy CNV 2.6 3. Create the following NodeNetworkConfigurationPolicy: --- apiVersion: nmstate.io/v1alpha1 kind: NodeNetworkConfigurationPolicy metadata: name: bigip-bridges spec: nodeSelector: node-role.kubernetes.io/worker-lb: "" desiredState: interfaces: - name: ens4f1.375 type: vlan state: up vlan: base-iface: ens4f1 id: 375 - name: bigip-mgmt description: Linux bridge with ens4f1 vlan375 as a port! type: linux-bridge state: up bridge: options: stp: enabled: false port: - name: ens4f1.375 - name: ens4f1.376 type: vlan state: up vlan: base-iface: ens4f1 id: 376 - name: bigip-ha description: Linux bridge with ens4f1 vlan376 as a port! type: linux-bridge state: up bridge: options: stp: enabled: false port: - name: ens4f1.376 Actual results: Configuration fails and br-ex bridge which provides external connectivity gets disabled. Expected results: Vlan interfaces and bridges get created and do not interrupt existing connections. Additional info: Attaching nmcli con, ip a output before and after applying NNCP, NetworkManager journal and nmstate-handler pod logs.