Bug 2177592

Summary: Cannot bring linux bridge and its port from deactivate to activate
Product: Red Hat Enterprise Linux 9 Reporter: Mingyu Shi <mshi>
Component: nmstateAssignee: Gris Ge <fge>
Status: CLOSED ERRATA QA Contact: Mingyu Shi <mshi>
Severity: medium Docs Contact:
Priority: medium    
Version: 9.2CC: ellorent, ferferna, jiji, jishi, network-qe, sfaye, till
Target Milestone: rcKeywords: Triaged
Target Release: ---Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: nmstate-2.2.9-1.el9 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-11-07 08:24:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2144442    

Description Mingyu Shi 2023-03-13 04:39:29 UTC
Description of problem:
When linux bridge and its port are deactivated in nmcli, failed to take them to `state: up` with nmstate.

Version-Release number of selected component (if applicable):
nmstate-2.2.7-1.el9.x86_64
nispor-1.2.10-1.el9.x86_64
NetworkManager-1.42.2-1.el9.x86_64
DISTRO=RHEL-9.2.0-20230307.10

How reproducible:
100%

Steps to Reproduce:
ip netns add client
ip netns add server
ip link add veth0_c type veth peer name veth0_c_p
ip link add veth0_s type veth peer name veth0_s_p
ip link set veth0_c_p netns client
ip link set veth0_s_p netns server
ip netns exec client ip link set lo up
ip netns exec client ip link set veth0_c_p up
ip netns exec client ip addr add 192.168.0.1/24 dev veth0_c_p
ip netns exec client ip addr add 2000::1/64 dev veth0_c_p
ip netns exec server ip  link set lo up
ip netns exec server ip  link set veth0_s_p up
ip netns exec server ip addr add 192.168.0.2/24 dev veth0_s_p
ip netns exec server ip -6 addr add 2000::2/64 dev veth0_s_p
nmcli device set veth0_c managed yes
nmcli device set veth0_s managed yes

cat << EOF > full.yaml
interfaces:
- name: br_test
  type: linux-bridge
  state: up
  accept-all-mac-addresses: false
  bridge:
    options:
      gc-timer: 29022
      group-addr: 01:80:C2:00:00:00
      group-forward-mask: 0
      group-fwd-mask: 0
      hash-max: 4096
      hello-timer: 0
      mac-ageing-time: 300
      multicast-last-member-count: 2
      multicast-last-member-interval: 100
      multicast-membership-interval: 26000
      multicast-querier: false
      multicast-querier-interval: 25500
      multicast-query-interval: 12500
      multicast-query-response-interval: 1000
      multicast-query-use-ifaddr: false
      multicast-router: auto
      multicast-snooping: true
      multicast-startup-query-count: 2
      multicast-startup-query-interval: 3125
      stp:
        enabled: false
        forward-delay: 15
        hello-time: 2
        max-age: 20
        priority: 32768
      vlan-protocol: 802.1q
    port:
    - name: veth0_c
      stp-hairpin-mode: false
      stp-path-cost: 100
      stp-priority: 32
    - name: veth0_s
      stp-hairpin-mode: false
      stp-path-cost: 100
      stp-priority: 32
  description: br_test
  ethtool:
    feature:
      highdma: true
      rx-gro: true
      rx-gro-list: false
      rx-udp-gro-forwarding: false
      tx-checksum-ip-generic: true
      tx-esp-segmentation: true
      tx-fcoe-segmentation: false
      tx-generic-segmentation: true
      tx-gre-csum-segmentation: true
      tx-gre-segmentation: true
      tx-gso-list: true
      tx-gso-partial: true
      tx-gso-robust: false
      tx-ipxip4-segmentation: true
      tx-ipxip6-segmentation: true
      tx-nocache-copy: false
      tx-scatter-gather-fraglist: true
      tx-sctp-segmentation: true
      tx-tcp-ecn-segmentation: true
      tx-tcp-mangleid-segmentation: true
      tx-tcp-segmentation: true
      tx-tcp6-segmentation: true
      tx-tunnel-remcsum-segmentation: true
      tx-udp-segmentation: true
      tx-udp_tnl-csum-segmentation: true
      tx-udp_tnl-segmentation: true
      tx-vlan-hw-insert: true
      tx-vlan-stag-hw-insert: true
  ipv4:
    enabled: true
    address:
    - ip: 192.168.0.254
      prefix-length: 24
    dhcp: false
  ipv6:
    enabled: true
    addr-gen-mode: eui64
    address:
    - ip: 2000::a
      prefix-length: 64
    - ip: fe80::a8cf:a8ff:fe2d:6a48
      prefix-length: 64
    autoconf: false
    dhcp: false
  lldp:
    enabled: false
  mac-address: AA:CF:A8:2D:6A:48
  max-mtu: 65535
  min-mtu: 68
  mptcp:
    address-flags: []
  mtu: 1500
  wait-ip: any
EOF

nmstatectl apply full.yaml

# deactivate them
echo "
interfaces:
- name: br_test
  type: linux-bridge
  state: down
- name: veth0_c
  type: veth
  state: down
- name: veth0_s
  type: veth
  state: down
" | nmstatectl apply

# activate again, failed
nmstatectl apply full.yaml

Actual results:
[2023-03-13T04:35:33Z INFO  nmstate::query_apply::net_state] Created checkpoint /org/freedesktop/NetworkManager/Checkpoint/96
[2023-03-13T04:35:33Z INFO  nmstate::iface] Include interface veth0_c to edit as its controller required so
[2023-03-13T04:35:33Z INFO  nmstate::iface] Include interface veth0_s to edit as its controller required so
[2023-03-13T04:35:33Z WARN  nmstate::ip] Ignoring IPv6 link local address fe80::a8cf:a8ff:fe2d:6a48/64
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Modifying connection UUID Some("12e61481-0a7e-4cba-a548-6fdc649ff838"), ID Some("br_test"), type Some("bridge") name Some("br_test")
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Modifying connection UUID Some("ceb82790-8f02-417b-9ae4-13e8fd818d6f"), ID Some("veth0_c"), type Some("802-3-ethernet") name Some("veth0_c")
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Modifying connection UUID Some("7bc68c6d-e19c-44da-9a4d-f4e3c93b8009"), ID Some("veth0_s"), type Some("802-3-ethernet") name Some("veth0_s")
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Activating connection 12e61481-0a7e-4cba-a548-6fdc649ff838: br_test/bridge
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Activating connection ceb82790-8f02-417b-9ae4-13e8fd818d6f: veth0_c/802-3-ethernet
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Activating connection 7bc68c6d-e19c-44da-9a4d-f4e3c93b8009: veth0_s/802-3-ethernet
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Deactivating connection ceb82790-8f02-417b-9ae4-13e8fd818d6f: veth0_c/802-3-ethernet
[2023-03-13T04:35:33Z INFO  nmstate::nm::query_apply::profile] Deactivating connection 7bc68c6d-e19c-44da-9a4d-f4e3c93b8009: veth0_s/802-3-ethernet
[2023-03-13T04:35:33Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:34Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:35Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:36Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:38Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Deactivating connection 12e61481-0a7e-4cba-a548-6fdc649ff838: br_test/bridge
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Modifying connection UUID Some("12e61481-0a7e-4cba-a548-6fdc649ff838"), ID Some("br_test"), type Some("bridge") name Some("br_test")
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Modifying connection UUID Some("ceb82790-8f02-417b-9ae4-13e8fd818d6f"), ID Some("veth0_c"), type Some("802-3-ethernet") name Some("veth0_c")
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Modifying connection UUID Some("7bc68c6d-e19c-44da-9a4d-f4e3c93b8009"), ID Some("veth0_s"), type Some("802-3-ethernet") name Some("veth0_s")
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Activating connection 12e61481-0a7e-4cba-a548-6fdc649ff838: br_test/bridge
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Activating connection ceb82790-8f02-417b-9ae4-13e8fd818d6f: veth0_c/802-3-ethernet
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Activating connection 7bc68c6d-e19c-44da-9a4d-f4e3c93b8009: veth0_s/802-3-ethernet
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Deactivating connection ceb82790-8f02-417b-9ae4-13e8fd818d6f: veth0_c/802-3-ethernet
[2023-03-13T04:35:39Z INFO  nmstate::nm::query_apply::profile] Deactivating connection 7bc68c6d-e19c-44da-9a4d-f4e3c93b8009: veth0_s/802-3-ethernet
[2023-03-13T04:35:39Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:40Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:41Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:42Z INFO  nmstate::query_apply::net_state] Retrying on: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'
[2023-03-13T04:35:43Z INFO  nmstate::query_apply::net_state] Rollbacked to checkpoint /org/freedesktop/NetworkManager/Checkpoint/96
NmstateError: VerificationError: Verification failure: br_test.interface.bridge.port desire '[{"name":"veth0_c","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32},{"name":"veth0_s","stp-hairpin-mode":false,"stp-path-cost":100,"stp-priority":32}]', current '[]'

Expected results:
No failure

Additional info:
This is a sequence of https://bugzilla.redhat.com/show_bug.cgi?id=2177590, as unnormal rollback caused bridge and ports deactivated, the later execution cannot take them back.

Comment 1 Gris Ge 2023-03-20 04:35:39 UTC
Patch posted to upstream: https://github.com/nmstate/nmstate/pull/2280

Comment 4 Mingyu Shi 2023-03-21 02:51:39 UTC
*** Bug 2176728 has been marked as a duplicate of this bug. ***

Comment 5 Gris Ge 2023-03-22 07:34:13 UTC
*** Bug 2179314 has been marked as a duplicate of this bug. ***

Comment 11 errata-xmlrpc 2023-11-07 08:24:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:6323