Bug 2019457
| Summary: | nmstate changes names of existing ovs-bridge ports when attaching a veth | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Radim Hrazdil <rhrazdil> | |
| Component: | nmstate | Assignee: | Radim Hrazdil <rhrazdil> | |
| Status: | CLOSED ERRATA | QA Contact: | Mingyu Shi <mshi> | |
| Severity: | unspecified | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 8.4 | CC: | ferferna, fge, jiji, jishi, mshi, network-qe, till | |
| Target Milestone: | rc | Keywords: | Triaged, ZStream | |
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
|
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | nmstate-1.2.0-0.1.alpha2.el8 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2034138 2034139 (view as bug list) | Environment: | ||
| Last Closed: | 2022-05-10 13:34:48 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2034138, 2034139 | |||
Upstream PR: https://github.com/nmstate/nmstate/pull/1724 Hi Mingyu, Please do pretest using copr repo of these PRs: * RHEL 8.4: https://github.com/nmstate/nmstate/pull/1738 * RHEL 8.5: https://github.com/nmstate/nmstate/pull/1737 * RHEL 8.6: https://github.com/nmstate/nmstate/pull/1724 This bug is requesting 8.4 zstream (RHEL 8.5 is also required so we don't have regression comparing to 8.4.z). Working on it Hi Radim, Could you please tell me how to reach the state *BEFORE* you applied desiredState? Or before you applied the new state, please show me something like: ip link nmstatectl show br-ex,(and other ovs stuff) nmcli con If I understand correctly, in the *BEFORE* stage, all the ovs devices were uncontrolled by NetworkManager, right? Then it appears that there's no chance for nmstate to configure it, as nmstate now cannot deal with a runtime ovs bridge. So I got confused that how you can successfully apply the new state... Thank you! Created attachment 1842977 [details] nmstatectl files Hello Mingyu Shi, The state before applying the desiredState is a clean OCP installation after the process of migration to OVN Kubernetes: https://docs.openshift.com/container-platform/4.9/networking/ovn_kubernetes_network_provider/migrate-from-openshift-sdn.html#nw-ovn-kubernetes-migration_migrate-from-openshift-sdn The OVS components are created during this process and also are managed by NetworkManager Tested on 8.6/8.5/8.4, take 8.4 as an example:
Tested with versions:
nmstate-1.0.5-0.20211119102720219917.pr1738.2.g8590e6d.el8.noarch
nispor-1.0.1-4.el8.x86_64
NetworkManager-1.30.0-13.el8_4.x86_64
openvswitch2.13-2.13.0-124.el8fdp.x86_64
nmcli con add type ovs-bridge con.interface-name br-ex
nmcli con add type ovs-port conn.interface-name br-ex master br-ex
nmcli con add type ovs-interface slave-type ovs-port conn.interface-name br-ex master br-ex ipv4.method manual ipv4.addresses 192.168.199.2/24
nmcli con add type ovs-port conn.interface-name eno1 master br-ex
nmcli conn add type ethernet conn.interface eno1 master eno1
echo "interfaces:
- name: br-ex
type: ovs-interface
state: up
- name: eno1
type: ethernet
state: up
- name: br-ex
type: ovs-bridge
state: up
bridge:
options:
fail-mode: ''
mcast-snooping-enable: false
rstp: false
stp: false
port:
- name: br-ex
- name: eno1
- name: ovn-veth
lldp:
enabled: false
- name: cnv-veth
type: veth
state: up
ipv4:
enabled: false
dhcp: false
ipv6:
enabled: false
autoconf: false
dhcp: false
veth:
peer: ovn-veth
- name: ovn-veth
type: veth
state: up
ipv4:
enabled: false
dhcp: false
ipv6:
enabled: false
autoconf: false
dhcp: false
veth:
peer: cnv-veth" | nmstatectl set -
With older version of nmstate, the result:
[17:28:42@kvm-04-guest29 ~]0# ovs-vsctl show
4bb50d34-88d1-4dc8-86cb-b5f97d8d18c0
Bridge br-ex
Port ovs-port-ovn-veth
Interface ovn-veth
type: system
Port ovs-port-eno1
Interface eno1
type: system
Port ovs-port-br-ex
Interface br-ex
type: internal
ovs_version: "2.13.5"
The result if using fixed version:
[18:58:10@kvm-04-guest29 ~]0# ovs-vsctl show
4bb50d34-88d1-4dc8-86cb-b5f97d8d18c0
Bridge br-ex
Port ovn-veth
Interface ovn-veth
type: system
Port eno1
Interface eno1
type: system
Port br-ex
Interface br-ex
type: internal
ovs_version: "2.13.5"
Hi Radim,
It seems that the current fix only removes "PREFIX-" in the port name, it meets your scenario, but the root cause "changing names of existing ovs-bridge ports" still exists.
Does it OK for you?
Here's an example:
For testing, I manually prefixed "ovsport_" in port name:
[17:03:47@mshi860 ~]0# ovs-vsctl show
95ae970e-5943-488f-a63b-f9a07cf7a963
Bridge br-ex
Port ovsport_ovspatch0
Interface ovspatch0
type: patch
options: {peer=ovspatch0_p}
Port ovsport_veth0
Interface veth0
type: system
Port ovsport_ovspatch0_p
Interface ovspatch0_p
type: patch
options: {peer=ovspatch0}
Port ovsport_br-ex
Interface br-ex
type: internal
ovs_version: "2.13.5"
After applied the new state, the port name changed(it removed the prefix part "ovsport_"):
[17:04:02@mshi860 ~]0# ovs-vsctl show
95ae970e-5943-488f-a63b-f9a07cf7a963
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Port veth0
Interface veth0
type: system
Port ovspatch0_p
Interface ovspatch0_p
type: patch
options: {peer=ovspatch0}
Port ovspatch0
Interface ovspatch0
type: patch
options: {peer=ovspatch0_p}
Port ovn-veth
Interface ovn-veth
type: system
ovs_version: "2.13.5"
nmstate-1.2.0-1.el8.x86_64
nispor-1.2.2-1.el8.x86_64
NetworkManager-1.36.0-0.3.el8.x86_64
openvswitch2.13-2.13.0-142.el8fdp.x86_64
Hello Mingyu, Thanks for the heads up. In our use-case, we're not using custom prefixes, so the current behaviour works for us. But I see your point that the BZ desicription is not being completely honest. Maybe we can change the title to "nmstate prefixes names of existing ovs-bridge ports with 'ovs-port-'" ? What do you think? In the future, a new BZ may be opened to achieve name consistency for manually prefixed interfaces, if the need arises. As https://bugzilla.redhat.com/show_bug.cgi?id=2019457#c13 shows, nmstate now uses same name on ovs bridge, port and interface. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:1772 |
Description of problem: When applying nmstate desiredState on OCP with OVNKubernetes that connects a veth interface to br-ex ovs-bridge, the names of ports of the ovs-bridge are prepended with 'ovs-port-'. State br-ex state before applying desiredState: $ ovs-vsctl show Bridge br-ex Port patch-br-ex_xyz.redhat.com-to-br-int Interface patch-br-ex_xyz.redhat.com-to-br-int type: patch options: {peer=patch-br-int-to-br-ex_xyz.redhat.com} Port br-ex Interface br-ex type: internal Port eno1 Interface eno1 type: system After applying desiredState $ ovs-vsctl show Bridge br-ex Port patch-br-ex_xyz.redhat.com-to-br-int Interface patch-br-ex_xyz.redhat.com-to-br-int type: patch options: {peer=patch-br-int-to-br-ex_xyz.redhat.com} Port ovs-port-ovn-veth Interface ovn-veth type: system Port ovs-port-br-ex Interface br-ex type: internal Port ovs-port-eno1 Interface eno1 type: system the desiredState used: interfaces: - name: br-ex type: ovs-interface state: up - name: eno1 type: ethernet state: up - name: br-ex type: ovs-bridge state: up bridge: options: fail-mode: '' mcast-snooping-enable: false rstp: false stp: false port: - name: br-ex - name: eno1 - name: ovn-veth lldp: enabled: false - name: cnv-veth type: veth state: up ipv4: enabled: false dhcp: false ipv6: enabled: false autoconf: false dhcp: false veth: peer: ovn-veth - name: ovn-veth type: veth state: up ipv4: enabled: false dhcp: false ipv6: enabled: false autoconf: false dhcp: false veth: peer: cnv-veth As a consequence, this seems to break ovnkube-node pods, as these pods are failing with the following error after applying the desiredState: I1101 14:47:29.362491 2671168 ovs.go:213] exec(16): stderr: "ovs-vsctl: no row \"ovs-port-eno1\" in table Interface\n" I1101 14:47:29.362495 2671168 ovs.go:215] exec(16): err: exit status 1 F1101 14:47:29.362522 2671168 ovnkube.go:130] Bridge for interface failed for br-ex: Failed to find intfName for br-ex: failed to get ofport of ovs-port-eno1, stderr: "ovs-vsctl: no row \"ovs-port-eno1\" in table Interface\n", error: exit status 1 Version-Release number of selected component (if applicable): nmstate 1.0.4 $ oc version Client Version: 4.9.0 Server Version: 4.9.5 Kubernetes Version: v1.22.0-rc.0+a44d0f0 How reproducible: 100 Steps to Reproduce: 1. Deploy OCP with OVNKubernetes 2. Apply desiredState given above 3. ovnkube-node pods are crashing (and other pods as well as a result) Expected results: nmstate shouldn't be changing names of existing ovs-bridge ports.