Bug 1861263

Summary: Failed to converting bond created by iproute to NetworkManager using nmstate
Product: Red Hat Enterprise Linux 8 Reporter: Gris Ge <fge>
Component: nmstateAssignee: Gris Ge <fge>
Status: CLOSED ERRATA QA Contact: Mingyu Shi <mshi>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.3CC: ferferna, jiji, jishi, network-qe, till
Target Milestone: rc   
Target Release: 8.4   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: nmstate-0.3.4-3.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-04 03:08:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Reproduce script
none
Reproduce script
none
verified.log none

Description Gris Ge 2020-07-28 07:55:15 UTC
Created attachment 1702619 [details]
Reproduce script

Description of problem:

When a bond is created using iproute tool, using nmstate to make is persistent
after reboot will fail. Showing the bond lose its subordinates.


Version-Release number of selected component (if applicable):
nmstate-0.3.4-1.el8

How reproducible:
100%

Steps to Reproduce:
1. sudo ./bug.sh
2.
3.

Actual results:

Got libnmstate.error.NmstateVerificationError

Expected results:

No failure

Additional info:

Comment 3 Mingyu Shi 2020-07-31 01:33:07 UTC
Hi Gris, I got a same error with either nmstate-0.3.4-3.el8.noarch or nmstate-0.3.4-1.el8.noarch, can you have a look:

The following is with 0.3.4-3:

[09:01:52@hpe-dl380pgen8-02-vm-9 ~]0# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 52:54:01:2c:e1:31 brd ff:ff:ff:ff:ff:ff
[09:02:02@hpe-dl380pgen8-02-vm-9 ~]0# # Create link
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link add dummy1 type dummy
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link add dummy2 type dummy
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# 
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# # Create bond
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link add bond1 type bond
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link set dev dummy1 master bond1
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link set dev dummy2 master bond1
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# 
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# # Set bond up
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link set up dummy1
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link set up dummy2
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# ip link set up bond1
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# 
[09:02:21@hpe-dl380pgen8-02-vm-9 ~]0# sleep 1
[09:02:22@hpe-dl380pgen8-02-vm-9 ~]0# 
[09:02:22@hpe-dl380pgen8-02-vm-9 ~]0# echo '
> ---
> interfaces:
> - name: eth1
>   type: ethernet
>   state: up
> - name: eth2
>   type: ethernet
>   state: up
> - name: bond99
>   type: bond
>   state: up
>   link-aggregation:
>     mode: active-backup
>     slaves:
>     - dummy1
>     - dummy2' | sudo nmstatectl set -
2020-07-31 09:02:25,539 root         WARNING  Failed to load plugin nmstate_plugin_ovsdb: No module named 'ovs'
Traceback (most recent call last):
  File "/bin/nmstatectl", line 11, in <module>
    load_entry_point('nmstate==0.3.4', 'console_scripts', 'nmstatectl')()
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 67, in main
    return args.func(args)
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 256, in apply
    args.save_to_disk,
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 289, in apply_state
    save_to_disk=save_to_disk,
  File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 69, in apply
    net_state = NetState(desired_state, current_state, save_to_disk)
  File "/usr/lib/python3.6/site-packages/libnmstate/net_state.py", line 40, in __init__
    save_to_disk,
  File "/usr/lib/python3.6/site-packages/libnmstate/ifaces/ifaces.py", line 106, in __init__
    self._pre_edit_validation_and_cleanup()
  File "/usr/lib/python3.6/site-packages/libnmstate/ifaces/ifaces.py", line 128, in _pre_edit_validation_and_cleanup
    self._validate_over_booked_slaves()
  File "/usr/lib/python3.6/site-packages/libnmstate/ifaces/ifaces.py", line 419, in _validate_over_booked_slaves
    f"Interface {iface.name} slave {slave_name} is "
libnmstate.error.NmstateValueError: Interface bond99 slave dummy1 is already enslaved by interface bond1
[09:02:25@hpe-dl380pgen8-02-vm-9 ~]1# nmcli dev
DEVICE  TYPE      STATE                   CONNECTION  
ens3    ethernet  connected               System ens3 
bond1   bond      connected (externally)  bond1       
dummy1  dummy     connected (externally)  dummy1      
dummy2  dummy     connected (externally)  dummy2      
lo      loopback  unmanaged               --          


After rebooting, NM profiles and devices were all removed

Comment 4 Gris Ge 2020-07-31 03:36:07 UTC
(In reply to Mingyu Shi from comment #3)
> Hi Gris, I got a same error with either nmstate-0.3.4-3.el8.noarch or
> nmstate-0.3.4-1.el8.noarch, can you have a look:

> 2020-07-31 09:02:25,539 root         WARNING  Failed to load plugin
> nmstate_plugin_ovsdb: No module named 'ovs'
> Traceback (most recent call last):
>   File "/bin/nmstatectl", line 11, in <module>
>     load_entry_point('nmstate==0.3.4', 'console_scripts', 'nmstatectl')()
>   File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 67,
> in main
>     return args.func(args)
>   File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line
> 256, in apply
>     args.save_to_disk,
>   File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line
> 289, in apply_state
>     save_to_disk=save_to_disk,
>   File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 69,
> in apply
>     net_state = NetState(desired_state, current_state, save_to_disk)
>   File "/usr/lib/python3.6/site-packages/libnmstate/net_state.py", line 40,
> in __init__
>     save_to_disk,
>   File "/usr/lib/python3.6/site-packages/libnmstate/ifaces/ifaces.py", line
> 106, in __init__
>     self._pre_edit_validation_and_cleanup()
>   File "/usr/lib/python3.6/site-packages/libnmstate/ifaces/ifaces.py", line
> 128, in _pre_edit_validation_and_cleanup
>     self._validate_over_booked_slaves()
>   File "/usr/lib/python3.6/site-packages/libnmstate/ifaces/ifaces.py", line
> 419, in _validate_over_booked_slaves
>     f"Interface {iface.name} slave {slave_name} is "
> libnmstate.error.NmstateValueError: Interface bond99 slave dummy1 is already
> enslaved by interface bond1

As the error message indicate, bond1 still exists holding the dummy1 and dummy2.
You need to mark bond1 as absent or remove dummy1 and dummy2 from its subordinate list.
You did mention bond1 in desire state, so nmstate is maintaining the current state
for bond1 which conflict with desired state.

And this bug is for converting bond created by ip route to NM managed.

Comment 5 Gris Ge 2020-07-31 03:39:58 UTC
(In reply to Gris Ge from comment #4)
> You did mention bond1 in desire state, so nmstate is maintaining the current
> state
> for bond1 which conflict with desired state.

Typo. Should be:

You didn't mention bond1 in desired state, so nmstate is using current state
for bond1 which is conflicting with desired state.

Comment 6 Gris Ge 2020-07-31 04:51:30 UTC
Created attachment 1703033 [details]
Reproduce script

Comment 7 Gris Ge 2020-07-31 08:25:40 UTC
Sorry the uploaded reproducer script is incorrect. The new one should works.

Comment 8 Mingyu Shi 2020-08-01 14:06:53 UTC
Created attachment 1703167 [details]
verified.log

Verified with versions:
nmstate-0.3.4-7.el8.noarch
NetworkManager-1.26.0-2.el8.x86_64
DISTRO=RHEL-8.3.0-20200728.n.0
Linux hpe-dl380pgen8-02-vm-9.hpe2.lab.eng.bos.redhat.com 4.18.0-228.el8.x86_64 #1 SMP Mon Jul 27 06:05:31 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Comment 11 errata-xmlrpc 2020-11-04 03:08:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4696