Bug 1869345

Summary:	Cannot remove all ovs ports when ovsdb is enabled
Product:	Red Hat Enterprise Linux 8	Reporter:	Mingyu Shi <mshi>
Component:	nmstate	Assignee:	Gris Ge <fge>
Status:	CLOSED ERRATA	QA Contact:	Mingyu Shi <mshi>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	8.3	CC:	ferferna, jiji, jishi, network-qe, till
Target Milestone:	rc	Flags:	pm-rhel: mirror+
Target Release:	8.3
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	nmstate-0.3.4-12.el8	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-11-04 03:08:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Mingyu Shi 2020-08-17 15:59:45 UTC

Description of problem:
Cannot remove all ovs ports when ovsdb is enabled

Version-Release number of selected component (if applicable):
nmstate-0.3.4-10.el8.noarch
NetworkManager-1.26.0-3.el8.x86_64
DISTRO=RHEL-8.3.0-20200811.0
Linux hp-dl388g8-04.rhts.eng.pek2.redhat.com 4.18.0-232.el8.x86_64 #1 SMP Mon Aug 10 06:55:47 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
openvswitch-selinux-extra-policy-1.0-22.el8fdp.noarch
openvswitch2.13-2.13.0-39.el8fdp.x86_64
python3-openvswitch2.13-2.13.0-39.el8fdp.x86_64


How reproducible:
100%

Steps to Reproduce:
# cat ovsbr-with-port.yaml 
---
interfaces:
  - name: ovs-br0
    type: ovs-bridge
    state: up
    bridge:
      port:
        - name: ovs0

# cat ovsbr-noport.yaml 
---
interfaces:
  - name: ovs-br0
    type: ovs-bridge
    state: up
    bridge:
      port: []
 
1. nmstatectl set ovsbr-with-port.yaml
2. nmstatectl set ovsbr-noport.yaml

Actual results:
Failed

Expected results:
No failure

Additional info:
[23:57:34@hp-dl388g8-04 ~]0# nmstatectl set ovsbr-noport.yaml 
2020-08-17 23:57:41,748 root         DEBUG    Async action: Create checkpoint started
2020-08-17 23:57:41,754 root         DEBUG    Checkpoint None created for all devices
2020-08-17 23:57:41,754 root         DEBUG    Async action: Create checkpoint finished
2020-08-17 23:57:41,756 root         DEBUG    Async action: Update profile: ovs-br0 started
2020-08-17 23:57:41,761 root         DEBUG    Async action: Update profile: ovs-br0 finished
2020-08-17 23:57:41,761 root         DEBUG    Async action: Reapply device config: ovs-br0 started
2020-08-17 23:57:41,764 root         DEBUG    Async action: Reapply device config: ovs-br0 finished
2020-08-17 23:57:41,764 root         DEBUG    Async action: Deactivate profile: ovs0 started
2020-08-17 23:57:41,764 root         DEBUG    Async action: Deactivate profile: ovs-port-ovs0 started
2020-08-17 23:57:41,770 root         DEBUG    Connection deactivation succeeded on ovs-port-ovs0
2020-08-17 23:57:41,770 root         DEBUG    Async action: Deactivate profile: ovs-port-ovs0 finished
2020-08-17 23:57:41,808 root         DEBUG    Connection deactivation succeeded on ovs0
2020-08-17 23:57:41,809 root         DEBUG    Async action: Deactivate profile: ovs0 finished
2020-08-17 23:57:41,809 root         DEBUG    Async action: Delete profile: id:ovs0, uuid:9bd06024-8189-4541-b829-8b1836e30bb4 started
2020-08-17 23:57:41,809 root         DEBUG    Async action: Delete profile: id:ovs-port-ovs0, uuid:46ec0298-d4be-45b6-a7d9-8508113eb7f2 started
2020-08-17 23:57:41,812 root         DEBUG    Async action: Delete profile: id:ovs0, uuid:9bd06024-8189-4541-b829-8b1836e30bb4 finished
2020-08-17 23:57:41,812 root         DEBUG    Async action: Delete profile: id:ovs-port-ovs0, uuid:46ec0298-d4be-45b6-a7d9-8508113eb7f2 finished
2020-08-17 23:57:41,813 root         DEBUG    Async action: Rollback to checkpoint /org/freedesktop/NetworkManager/Checkpoint/5 started
2020-08-17 23:57:41,825 root         DEBUG    Checkpoint /org/freedesktop/NetworkManager/Checkpoint/5 rollback executed
2020-08-17 23:57:41,825 root         DEBUG    Interface eno4 rollback succeeded
2020-08-17 23:57:41,825 root         DEBUG    Interface ens2f1 rollback succeeded
2020-08-17 23:57:41,826 root         DEBUG    Interface ens6f1 rollback succeeded
2020-08-17 23:57:41,826 root         DEBUG    Async action: Waiting for rolling back ovs-br0 started
2020-08-17 23:57:41,826 root         DEBUG    Interface ovs-br0 rollback succeeded
2020-08-17 23:57:41,826 root         DEBUG    Interface ens6f2 rollback succeeded
2020-08-17 23:57:41,826 root         DEBUG    Interface eno3 rollback succeeded
2020-08-17 23:57:41,826 root         DEBUG    Interface /org/freedesktop/NetworkManager/Devices/21 rollback succeeded
2020-08-17 23:57:41,827 root         DEBUG    Interface veth1 rollback succeeded
2020-08-17 23:57:41,827 root         DEBUG    Interface veth0 rollback succeeded
2020-08-17 23:57:41,827 root         DEBUG    Interface /org/freedesktop/NetworkManager/Devices/22 rollback succeeded
2020-08-17 23:57:41,827 root         DEBUG    Interface eno1 rollback succeeded
2020-08-17 23:57:41,827 root         DEBUG    Interface ens6f3 rollback succeeded
2020-08-17 23:57:41,827 root         DEBUG    Interface veth1_ep rollback succeeded
2020-08-17 23:57:41,827 root         DEBUG    Interface veth0_ep rollback succeeded
2020-08-17 23:57:41,828 root         DEBUG    Interface ens6f0 rollback succeeded
2020-08-17 23:57:41,828 root         DEBUG    Interface lo rollback succeeded
2020-08-17 23:57:41,828 root         DEBUG    Interface ens2f0 rollback succeeded
2020-08-17 23:57:41,828 root         DEBUG    Interface veth2_ep rollback succeeded
2020-08-17 23:57:41,828 root         DEBUG    Interface veth2 rollback succeeded
2020-08-17 23:57:41,828 root         DEBUG    Interface eno2 rollback succeeded
2020-08-17 23:57:41,828 root         DEBUG    Async action: Rollback to checkpoint /org/freedesktop/NetworkManager/Checkpoint/5 finished
2020-08-17 23:57:41,834 root         DEBUG    Active connection of device None has been replaced
2020-08-17 23:57:41,838 root         DEBUG    Connection activation succeeded: dev=ovs-br0, con-state=<enum NM_ACTIVE_CONNECTION_STATE_ACTIVATING of type NM.ActiveConnectionState>, dev-state=<enum NM_DEVICE_STATE_IP_CONFIG of type NM.DeviceState>, state-flags=<flags NM_ACTIVATION_STATE_FLAG_IS_MASTER | NM_ACTIVATION_STATE_FLAG_LAYER2_READY of type NM.ActivationStateFlags>
2020-08-17 23:57:41,838 root         DEBUG    Async action: Waiting for rolling back ovs-br0 finished
Traceback (most recent call last):
  File "/usr/bin/nmstatectl", line 11, in <module>
    load_entry_point('nmstate==0.3.4', 'console_scripts', 'nmstatectl')()
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 67, in main
    return args.func(args)
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 256, in apply
    args.save_to_disk,
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 289, in apply_state
    save_to_disk=save_to_disk,
  File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 71, in apply
    _apply_ifaces_state(plugins, net_state, verify_change, save_to_disk)
  File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 104, in _apply_ifaces_state
    plugin.apply_changes(net_state, save_to_disk)
  File "/usr/lib/python3.6/site-packages/libnmstate/plugins/nmstate_plugin_ovsdb.py", line 219, in apply_changes
    self._db_write(pending_changes)
  File "/usr/lib/python3.6/site-packages/libnmstate/plugins/nmstate_plugin_ovsdb.py", line 235, in _db_write
    f"BUG: row {new_rows} does not exists in OVS DB "
libnmstate.error.NmstatePluginError: BUG: row {'ovs-br0'} does not exists in OVS DB and currently we don't create new row

Comment 4 Gris Ge 2020-08-18 09:59:33 UTC

BTW.

The OpenvSwitch does not allows a OVS bridge holding no interface, please be aware you cannot modify ovsdb for a empty OVS bridge.

Comment 5 Mingyu Shi 2020-08-21 03:26:30 UTC

Verified with versions:
nmstate-0.3.4-12.el8.noarch
NetworkManager-1.26.0-3.el8.x86_64
DISTRO=RHEL-8.3.0-20200811.0
Linux hp-dl380pg8-12.rhts.eng.pek2.redhat.com 4.18.0-232.el8.x86_64 #1 SMP Mon Aug 10 06:55:47 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
openvswitch-selinux-extra-policy-1.0-22.el8fdp.noarch
openvswitch2.13-2.13.0-39.el8fdp.x86_64
python3-openvswitch2.13-2.13.0-39.el8fdp.x86_64


[11:25:04@hp-dl380pg8-12 ~]0# nmstatectl set ovsbr-with-port.yaml 
2020-08-21 11:25:09,956 root         DEBUG    Async action: Create checkpoint started
2020-08-21 11:25:09,963 root         DEBUG    Checkpoint None created for all devices
2020-08-21 11:25:09,963 root         DEBUG    Async action: Create checkpoint finished
2020-08-21 11:25:09,966 root         DEBUG    Async action: Add profile: ovs-br0 started
2020-08-21 11:25:09,968 root         DEBUG    Async action: Add profile: ovs0 started
2020-08-21 11:25:09,970 root         DEBUG    Async action: Add profile: ovs-port-ovs0 started
2020-08-21 11:25:09,978 root         DEBUG    Async action: Add profile: ovs-br0 finished
2020-08-21 11:25:09,985 root         DEBUG    Async action: Add profile: ovs0 finished
2020-08-21 11:25:09,985 root         DEBUG    Async action: Add profile: ovs-port-ovs0 finished
2020-08-21 11:25:09,986 root         DEBUG    Async action: Activate profile: ovs-br0 started
2020-08-21 11:25:09,993 root         DEBUG    Connection activation initiated: dev=ovs-br0, con-state=<enum NM_ACTIVE_CONNECTION_STATE_ACTIVATING of type NM.ActiveConnectionState>
2020-08-21 11:25:10,007 root         DEBUG    Connection activation succeeded: dev=ovs-br0, con-state=<enum NM_ACTIVE_CONNECTION_STATE_ACTIVATING of type NM.ActiveConnectionState>, dev-state=<enum NM_DEVICE_STATE_IP_CONFIG of type NM.DeviceState>, state-flags=<flags NM_ACTIVATION_STATE_FLAG_IS_MASTER | NM_ACTIVATION_STATE_FLAG_LAYER2_READY | NM_ACTIVATION_STATE_FLAG_MASTER_HAS_SLAVES of type NM.ActivationStateFlags>
2020-08-21 11:25:10,008 root         DEBUG    Async action: Activate profile: ovs-br0 finished
2020-08-21 11:25:10,008 root         DEBUG    Async action: Activate profile: ovs-port-ovs0 started
2020-08-21 11:25:10,022 root         DEBUG    Connection activation initiated: dev=ovs-port-ovs0, con-state=<enum NM_ACTIVE_CONNECTION_STATE_ACTIVATING of type NM.ActiveConnectionState>
2020-08-21 11:25:10,056 root         DEBUG    Connection activation succeeded: dev=ovs-port-ovs0, con-state=<enum NM_ACTIVE_CONNECTION_STATE_ACTIVATING of type NM.ActiveConnectionState>, dev-state=<enum NM_DEVICE_STATE_IP_CONFIG of type NM.DeviceState>, state-flags=<flags NM_ACTIVATION_STATE_FLAG_IS_MASTER | NM_ACTIVATION_STATE_FLAG_IS_SLAVE | NM_ACTIVATION_STATE_FLAG_LAYER2_READY of type NM.ActivationStateFlags>
2020-08-21 11:25:10,057 root         DEBUG    Async action: Activate profile: ovs-port-ovs0 finished
2020-08-21 11:25:10,057 root         DEBUG    Async action: Activate profile: ovs0 started
2020-08-21 11:25:10,065 root         DEBUG    Connection activation initiated: dev=ovs0, con-state=<enum NM_ACTIVE_CONNECTION_STATE_ACTIVATING of type NM.ActiveConnectionState>
2020-08-21 11:25:10,624 root         DEBUG    Connection activation succeeded: dev=ovs0, con-state=<enum NM_ACTIVE_CONNECTION_STATE_ACTIVATED of type NM.ActiveConnectionState>, dev-state=<enum NM_DEVICE_STATE_ACTIVATED of type NM.DeviceState>, state-flags=<flags NM_ACTIVATION_STATE_FLAG_IS_SLAVE | NM_ACTIVATION_STATE_FLAG_LAYER2_READY | NM_ACTIVATION_STATE_FLAG_IP4_READY | NM_ACTIVATION_STATE_FLAG_IP6_READY of type NM.ActivationStateFlags>
2020-08-21 11:25:10,625 root         DEBUG    Async action: Activate profile: ovs0 finished
2020-08-21 11:25:10,710 root         DEBUG    Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/1 started
2020-08-21 11:25:10,714 root         DEBUG    Checkpoint /org/freedesktop/NetworkManager/Checkpoint/1 destroyed
2020-08-21 11:25:10,714 root         DEBUG    Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/1 finished
Desired state applied: 
---
interfaces:
- name: ovs-br0
  type: ovs-bridge
  state: up
  bridge:
    port:
    - name: ovs0
[11:25:10@hp-dl380pg8-12 ~]0# nmstatectl set ovsbr-noport.yaml 
2020-08-21 11:25:15,997 root         DEBUG    Async action: Create checkpoint started
2020-08-21 11:25:16,006 root         DEBUG    Checkpoint None created for all devices
2020-08-21 11:25:16,006 root         DEBUG    Async action: Create checkpoint finished
2020-08-21 11:25:16,011 root         DEBUG    Async action: Update profile: ovs-br0 started
2020-08-21 11:25:16,110 root         DEBUG    Async action: Update profile: ovs-br0 finished
2020-08-21 11:25:16,112 root         DEBUG    Async action: Reapply device config: ovs-br0 started
2020-08-21 11:25:16,116 root         DEBUG    Async action: Reapply device config: ovs-br0 finished
2020-08-21 11:25:16,116 root         DEBUG    Async action: Deactivate profile: ovs0 started
2020-08-21 11:25:16,117 root         DEBUG    Async action: Deactivate profile: ovs-port-ovs0 started
2020-08-21 11:25:16,140 root         DEBUG    Connection deactivation succeeded on ovs-port-ovs0
2020-08-21 11:25:16,140 root         DEBUG    Async action: Deactivate profile: ovs-port-ovs0 finished
2020-08-21 11:25:16,247 root         DEBUG    Connection deactivation succeeded on ovs0
2020-08-21 11:25:16,247 root         DEBUG    Async action: Deactivate profile: ovs0 finished
2020-08-21 11:25:16,248 root         DEBUG    Async action: Delete profile: id:ovs0, uuid:7a552667-8f7a-427f-86e9-a37e456ecb0e started
2020-08-21 11:25:16,248 root         DEBUG    Async action: Delete profile: id:ovs-port-ovs0, uuid:6c4be3e5-2925-4bd4-8e80-c37ff8c04097 started
2020-08-21 11:25:16,253 root         DEBUG    Async action: Delete profile: id:ovs0, uuid:7a552667-8f7a-427f-86e9-a37e456ecb0e finished
2020-08-21 11:25:16,254 root         DEBUG    Async action: Delete profile: id:ovs-port-ovs0, uuid:6c4be3e5-2925-4bd4-8e80-c37ff8c04097 finished
2020-08-21 11:25:16,255 root         DEBUG    Async action: Delete device: ovs0 started
2020-08-21 11:25:16,257 root         DEBUG    Interface is not real anymore: iface=ovs0
2020-08-21 11:25:16,257 root         DEBUG    Ignored error: g-dbus-error-quark: No such interface 'org.freedesktop.NetworkManager.Device' on object at path /org/freedesktop/NetworkManager/Devices/11 (19)
2020-08-21 11:25:16,257 root         DEBUG    Async action: Delete device: ovs0 finished
2020-08-21 11:25:16,413 root         DEBUG    Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/2 started
2020-08-21 11:25:16,418 root         DEBUG    Checkpoint /org/freedesktop/NetworkManager/Checkpoint/2 destroyed
2020-08-21 11:25:16,419 root         DEBUG    Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/2 finished
Desired state applied: 
---
interfaces:
- name: ovs-br0
  type: ovs-bridge
  state: up
  bridge:
    port: []
[11:25:16@hp-dl380pg8-12 ~]0# nmstatectl show 'ovs*'
---
dns-resolver:
  config:
    search: []
    server: []
  running:
    search:
    - rhts.eng.pek2.redhat.com
    server:
    - 10.73.2.107
    - 10.73.2.108
    - 10.66.127.10
route-rules:
  config: []
routes:
  config: []
  running: []
interfaces:
- name: ovs-br0
  type: ovs-bridge
  state: up
  bridge:
    options:
      fail-mode: ''
      mcast-snooping-enable: false
      rstp: false
      stp: false
    port: []
  lldp:
    enabled: false

Comment 8 errata-xmlrpc 2020-11-04 03:08:06 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4696