Bug 2249766

Description of problem:
While trying to create 2 different ovs_bridges with 2 vlan from the same PHY bond, os-net-config (with nmstate provider) failed with the following error message:

os_net_config.impl_nmstate:Overall rules\nroute-rules:\n  config: []\n\nDEBUG:os_net_config.impl_nmstate:----------------------------\nDEBUG:os_net_config.impl_nmstate:Overall DNS\ndns-resolver:\n  config:\n    search: []\n    server: []\n\nDEBUG:os_net_config.impl_nmstate:----------------------------\nDEBUG:os_net_config.impl_nmstate:Applying the config with nmstate\ndns-resolver:\n  config:\n    search: []\n    server: []\ninterfaces:\n- ethernet: {}\n  ipv4:\n    auto-dns: true\n    auto-gateway: true\n    auto-routes: true\n    dhcp: true\n    enabled: true\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: eno1\n  state: up\n  type: ethernet\n- ethernet: {}\n  ipv4:\n    address:\n    - ip: 192.168.91.18\n      prefix-length: 24\n    dhcp: false\n    enabled: true\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: eno2\n  state: up\n  type: ethernet\n- ethernet: {}\n  ipv4:\n    dhcp: false\n    enabled: false\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: eno3\n  state: up\n  type: ethernet\n- ethernet: {}\n  ipv4:\n    dhcp: false\n    enabled: false\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: eno4\n  state: up\n  type: ethernet\n- ipv4:\n    dhcp: false\n    enabled: false\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: br-tenant-1-p\n  state: up\n  type: ovs-interface\n- ipv4:\n    dhcp: false\n    enabled: false\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  link-aggregation:\n    mode: 802.3ad\n    port:\n    - eno3\n    - eno4\n  mtu: 9000\n  name: bond_erog\n  state: up\n  type: bond\n- ipv4:\n    dhcp: false\n    enabled: false\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: br-tenant-2-p\n  state: up\n  type: ovs-interface\n- bridge:\n    options:\n      fail-mode: standalone\n      mcast-snooping-enable: false\n      rstp: false\n      stp: false\n    port:\n    - name: bond_erog\n    - name: vlan508\n      vlan:\n        mode: access\n        tag: 508\n    - name: br-tenant-1-p\n  name: br-tenant-1\n  ovs-db:\n    external_ids: {}\n    other_config: {}\n  state: up\n  type: ovs-bridge\n- bridge:\n    options:\n      fail-mode: standalone\n      mcast-snooping-enable: false\n      rstp: false\n      stp: false\n    port:\n    - name: bond_erog\n    - name: vlan509\n      vlan:\n        mode: access\n        tag: 509\n    - name: br-tenant-2-p\n  name: br-tenant-2\n  ovs-db:\n    external_ids: {}\n    other_config: {}\n  state: up\n  type: ovs-bridge\n- ipv4:\n    address:\n    - ip: 152.50.32.41\n      prefix-length: 24\n    dhcp: false\n    enabled: true\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: vlan504\n  state: up\n  type: vlan\n  vlan:\n    base-iface: eno2\n    id: 504\n- ipv4:\n    address:\n    - ip: 152.50.16.188\n      prefix-length: 24\n    dhcp: false\n    enabled: true\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: vlan502\n  state: up\n  type: vlan\n  vlan:\n    base-iface: eno2\n    id: 502\n- ipv4:\n    address:\n    - ip: 192.168.160.13\n      prefix-length: 24\n    dhcp: false\n    enabled: true\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: vlan508\n  state: up\n  type: ovs-interface\n- ipv4:\n    address:\n    - ip: 192.178.170.13\n      prefix-length: 24\n    dhcp: false\n    enabled: true\n  ipv6:\n    autoconf: false\n    dhcp: false\n    enabled: false\n  name: vlan509\n  state: up\n  type: ovs-interface\nroute-rules:\n  config: []\nroutes:\n  config:\n  - destination: 169.254.169.254/32\n    next-hop-address: 192.168.91.18\n    next-hop-interface: eno2\n  - destination: 0.0.0.0/0\n    next-hop-address: 192.168.91.18\n    next-hop-interface: eno2\n\nWARNING:root:1699874684:nispor::query::ethtool:/builddir/.cargo/registry/src/github.com-1ecc6299db9ec823/nispor-1.2.14/query/ethtool.rs:480: WARN: Unsupported EthtoolCoalesceAttr Other(DefaultNla { kind: 24, value: [0] })\nWARNING:root:1699874684:nispor::query::ethtool:/builddir/.cargo/registry/src/github.com-1ecc6299db9ec823/nispor-1.2.14/query/ethtool.rs:480: WARN: Unsupported EthtoolCoalesceAttr Other(DefaultNla { kind: 25, value: [1] })\nWARNING:root:1699874684:nispor::query::ethtool:/builddir/.cargo/registry/src/github.com-1ecc6299db9ec823/nispor-1.2.14/query/ethtool.rs:480: WARN: Unsupported EthtoolCoalesceAttr Other(DefaultNla { kind: 24, value: [0] })\nWARNING:root:1699874684:nispor::query::ethtool:/builddir/.cargo/registry/src/github.com-1ecc6299db9ec823/nispor-1.2.14/query/ethtool.rs:480: WARN: Unsupported EthtoolCoalesceAttr Other(DefaultNla { kind: 25, value: [1] })\nWARNING:root:1699874684:nispor::query::ethtool:/builddir/.cargo/registry/src/github.com-1ecc6299db9ec823/nispor-1.2.14/query/ethtool.rs:523: WARN: Unsupported EthtoolRingAttr TcpDataSplit(1)\nWARNING:root:1699874684:nispor::query::ethtool:/builddir/.cargo/registry/src/github.com-1ecc6299db9ec823/nispor-1.2.14/query/ethtool.rs:523: WARN: Unsupported EthtoolRingAttr TcpDataSplit(1)\nERROR:root:1699874684:nmstate::ifaces::inter_ifaces_controller:src/lib/ifaces/inter_ifaces_controller.rs:23: InvalidArgument: Port bond_erog is overbooked by two controller: br-tenant-2, br-tenant-1\nTraceback (most recent call last):\n  File \"/bin/os-net-config\", line 10, in <module>\n    sys.exit(main())\n  File \"/usr/lib/python3.9/site-packages/os_net_config/cli.py\", line 346, in main\n    files_changed = provider.apply(cleanup=opts.cleanup,\n  File \"/usr/lib/python3.9/site-packages/os_net_config/impl_nmstate.py\", line 1619, in apply\n    self.nmstate_apply(apply_data, verify=True)\n  File \"/usr/lib/python3.9/site-packages/os_net_config/impl_nmstate.py\", line 490, in nmstate_apply\n    netapplier.apply(new_state, verify_change=verify)\n  File \"/usr/lib/python3.9/site-packages/libnmstate/netapplier.py\", line 29, in apply\n    return apply_net_state(\n  File \"/usr/lib/python3.9/site-packages/libnmstate/clib_wrapper.py\", line 127, in apply_net_state\n    raise map_error(err_kind, err_msg)\nlibnmstate.error.NmstateValueError: Port bond_erog is overbooked by two controller: br-tenant-2, br-tenant-1\n",


Version-Release number of selected component (if applicable):
(undercloud) [stack@undercloud-0 ~]$ cat core_puddle_version 
RHOS-17.1-RHEL-9-20230925.n.1(undercloud)

How reproducible:
Always

Steps to Reproduce:
1. Set "nmstate" as provider for os-net-config
2. Try to do node provision with following nic template:

- type: linux_bond
  name: bond_erog
  use_dhcp: false
  bonding_options: "mode=802.3ad"
  mtu: 9000
  members:
  - type: interface
    name: nic3
    primary: true
  - type: interface
    name: nic4

- type: ovs_bridge
  name: br-tenant-1
  mtu: 1350
  use_dhcp: false
  routes: []
  members:
  - type: interface
    name: bond_erog.508
    addresses:
    - ip_netmask: 163.162.31.101/28

- type: ovs_bridge
  name: br-tenant-2
  mtu: 1350
  use_dhcp: false
  members:
  - type: interface
    name: bond_erog.509
    addresses:
    - ip_netmask: 163.162.21.201/28

3. 

Actual results:
Node provision fails

Expected results:
Node provision should be successful 


Additional info: