Bug 1400891

Created attachment 1227221 [details]
Screenshot of setup bond0 in default DHCP mode

Created attachment 1227222 [details]
Screenshot of error,  bond0 's  primary slave NIC eno1 is independent

Created attachment 1227223 [details]
Screenshot of error,  bond0 shows no carrier, and only has 1 slave NIC eno2

Tested setting bond with NMTUI in redhat-virtualization-host-4.0-20161206.0, can set up bond under certain conditions.

1. Failed to setup bond over two NICs directly, due to NMTUI will setup another ifcfg for the same NIC, that will cause setup bond fail.

2. Only follow the below workaround steps, can setup bond via NMTUI successful.
Test steps(take NICs eno1 and eno2 for example):
1) eno1 is ON(can get ip 10.73.130.225), eno2 is OFF
2) Delete /etc/sysconfig/network-scripts/ifcfg-eno1 and /etc/sysconfig/network-scripts/ifcfg-eno2
3) Setup bond1 over eno1(primary) and eno2 via NMTUI.(Note: ifcfg and device name must be bond1, eno1 and eno2 are also configured like this)
4) bond1 can not get DHCP ip currently
5) Click "ON" of eno2 via cockpit, then bond1 can get DHCP ip(10.66.128.130), but now slave eno1 still have DHCP ip 10.73.130.225.
6) Reboot RHVH, then eno1's DHCP ip disappeared, bond1 can get DHCP ip 10.66.130.225.

7) Change to IP Manual mode for bond1, can be successful
8) Add RHVH to RHVM successful, but bond1, eno1, eno2 and ovirtmgmt show as Unmanaged Interfaces in cockpit.

According to above steps and results, I think setup bond via NMTUI also have issue.

Created attachment 1228926 [details]
All logs of Comment 4

We have to escalate this bug if we are going to enable cockpit network UI during 4.0.6.

Moving to NM b/c of comment 4 and set to urgent, because it's urgent to RHV.

Hi,

Logs from comment 5 show that after eno1 and eno2 are enslaved to the
bond, NM start DHCP at 03:35:50 and after no response from the server,
a rollback of the configuration (likely scheduled by cockpit) kicks in
at 03:36:02, restoring the old configuration with DHCP on eno1.

To better isolate the problem, can you please reproduce the problem
with nmtui alone and describe the necessary steps (including which
connections you create and their parameters)?

Also, before doing that, would you please run 'nmcli general logging
level trace' as root (note that this setting is not persistent and so
if you reboot or restart NM you should execute the command again) to
increase the logging level.

After the failure in the bond setup happens, please attach system
logs. Thanks!

Test version:
redhat-virtualization-host-4.0-20161206.0
NetworkManager-1.4.0-13.el7_3.x86_64

Test goal: Setup bond1(active-backup mode) over two NICs eno1 and eno2.

Test steps:
1. eno1 is ON(can get ip 10.73.130.225), eno2 is OFF
2. run 'nmcli general logging level trace' as root
3. Delete /etc/sysconfig/network-scripts/ifcfg-eno1 and /etc/sysconfig/network-scripts/ifcfg-eno2
4. Setup bond1(active-backup mode) over eno1(primary) and eno2 via NMTUI.
(Note: Profile name and device must be bond1, eno1 and eno2 are also configured like this. Please refer to attachment screenshot for detailed configuration)

5. bond1 can get DHCP ip(10.73.128.240) currently, but slave eno1 still has DHCP ip(10.73.130.225).

# ip a s
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 08:94:ef:21:c0:4d brd ff:ff:ff:ff:ff:ff
    inet 10.73.130.225/23 brd 10.73.131.255 scope global dynamic eno1
       valid_lft 41269sec preferred_lft 41269sec
    inet6 2620:52:0:4982:33e8:e98a:268f:769b/64 scope global noprefixroute dynamic 
       valid_lft 2591861sec preferred_lft 604661sec
    inet6 fe80::a0de:27e5:b350:6b4d/64 scope link 
       valid_lft forever preferred_lft forever
3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond1 state UP qlen 1000
    link/ether 08:94:ef:21:c0:4e brd ff:ff:ff:ff:ff:ff
9: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 08:94:ef:21:c0:4e brd ff:ff:ff:ff:ff:ff
    inet 10.73.128.240/24 brd 10.73.128.255 scope global dynamic bond1
       valid_lft 1219sec preferred_lft 1219sec
    inet6 2620:52:0:4980:3776:eeef:63ea:5ca7/64 scope global noprefixroute dynamic 
       valid_lft 2591560sec preferred_lft 604360sec
    inet6 fe80::6ef8:66e2:a450:d697/64 scope link 
       valid_lft forever preferred_lft forever

Expected results:
1. In step3, do not have to delete original ifcfg-interface(slave NIC should not create new ifcfg-interface-1 file when create bond)
2. In step5, slave eno1 should not have DHCP ip, only bond1 has DHCP ip

Created attachment 1231976 [details]
Comment 11: All logs and all files in /etc/sysconfig/network-scripts/

Created attachment 1231977 [details]
comment 11: Screenshot of adding bond1

Created attachment 1231979 [details]
comment 11: screenshot of adding slave eno1 to bond1

Created attachment 1231980 [details]
comment 11: screenshot of adding slave eno2 to bond1

(In reply to Huijuan Zhao from comment #11)
> Test version:
> redhat-virtualization-host-4.0-20161206.0
> NetworkManager-1.4.0-13.el7_3.x86_64
>
> Test goal: Setup bond1(active-backup mode) over two NICs eno1 and eno2.
>
> Test steps:
> 1. eno1 is ON(can get ip 10.73.130.225), eno2 is OFF
> 2. run 'nmcli general logging level trace' as root
> 3. Delete /etc/sysconfig/network-scripts/ifcfg-eno1 and
> /etc/sysconfig/network-scripts/ifcfg-eno2

NM doesn't monitor files in /etc/sysconfig/network-scripts for
changes, so even if you delete a file there NM will still have the
connection in memory. Every time a change is made to a connection
file, a 'nmcli connection reload' must be performed to make NM pick up
the new connections.

Or, better, instead of manually deleting the file, you can delete the
connection through NM with:

 nmcli connection delete eno1 eno2

> 4. Setup bond1(active-backup mode) over eno1(primary) and eno2 via NMTUI.
> (Note: Profile name and device must be bond1, eno1 and eno2 are also
> configured like this. Please refer to attachment screenshot for detailed
> configuration)
>
> 5. bond1 can get DHCP ip(10.73.128.240) currently, but slave eno1 still has
> DHCP ip(10.73.130.225).

That's because there are now two 'eno1' connections: the old one with
DHCP and the new bond slave; and the old eno1 connection is still active here.

A 'nmcli connection' command would probably show something like:

  # nmcli connection
  NAME                      UUID                                  TYPE            DEVICE
  bond1                     a3743ad6-2ac0-4bdf-961b-9103871ed7b4  bond            bond1
  eno1                      52f610b7-aae6-49c0-8adf-7fdc641c1422  802-3-ethernet  eno1
  eno2                      c0631c81-5381-45ab-9035-371e9478808a  802-3-ethernet  eno2
  eno1                      bd5106c6-d97c-43e2-a356-8392fa450d49  802-3-ethernet  --

and an inspection of the active eno1 connection would reveal that it's
the DHCP one:

 # nmcli connection show 52f610b7-aae6-49c0-8adf-7fdc641c1422
 [...]
 connection.master:                      --
 [...]
 ipv4.method:                            auto
 [...]

> Expected results:
> 1. In step3, do not have to delete original ifcfg-interface(slave NIC should
> not create new ifcfg-interface-1 file when create bond)
> 2. In step5, slave eno1 should not have DHCP ip, only bond1 has DHCP ip

I think both of these will disappear if the old connection is removed
as described above.

Please repeat the test adding a 'nmcli connection reload' after step 3
(or, in alternative, replace step 3 with 'nmcli connection delete eno1 eno2').

> I think both of these will disappear if the old connection is removed
> as described above.
> 
> Please repeat the test adding a 'nmcli connection reload' after step 3
> (or, in alternative, replace step 3 with 'nmcli connection delete eno1
> eno2').

Yes, you are right. 
I replaced step 3 with 'nmcli connection delete eno1 eno2', then only bond1 can get DHCP ip successfully. The result is what we expected.

Thanks for your detail instruction. 
So this is not NM issue, but cockpit call NM related issue, just setup bond failed via cockpit.

(In reply to Huijuan Zhao from comment #17)
> > I think both of these will disappear if the old connection is removed
> > as described above.
> > 
> > Please repeat the test adding a 'nmcli connection reload' after step 3
> > (or, in alternative, replace step 3 with 'nmcli connection delete eno1
> > eno2').
> 
> Yes, you are right. 
> I replaced step 3 with 'nmcli connection delete eno1 eno2', then only bond1
> can get DHCP ip successfully. The result is what we expected.
> 
> Thanks for your detail instruction. 
> So this is not NM issue, but cockpit call NM related issue, just setup bond
> failed via cockpit.

Since the issue seems not related to NM, can you please update the product/component of the BZ? (It seems I'm unable to do so due to bugzilla complaining about missing fields). Thanks!

Fabian, according to comment 16, 17, 18, could you please update the product/component of the BZ? 
Thanks!

Guys, this report sounds very similar to BZ 1395108 which Fixed In Version: cockpit-126-1.el7

Huijuan, do you agree that this bug is a dupe of BZ 1395108?

(In reply to Fabian Deutsch from comment #21)
> Huijuan, do you agree that this bug is a dupe of BZ 1395108?

Fabian, yes, I agree, and the test steps are almost same.

I also tested the below two scenarios, both work well.
1. Setup bond0 over two slave NICs which were OFF, after setup bond0, enable bond0 and slave NICs, bond0 works well, bond0 can get DHCP IP and ifcfg-${if} files are right.
2. Setup bond0 over two slave NICs which were ON, after setup bond0,  bond0 works well, bond0 can get DHCP IP and ifcfg-${if} files are right.

Thanks Huijuan and Michael, closing as a dupe acording to comment 22.

*** This bug has been marked as a duplicate of bug 1395108 ***