Bug 1722449
| Summary: | Anaconda network team configuration tests failing with rhel 8.1 nightlies ("failed to update config for port") | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Radek Vykydal <rvykydal> | ||||||
| Component: | libteam | Assignee: | Xin Long <lxin> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | LiLiang <liali> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | medium | ||||||||
| Version: | 8.0 | CC: | atragler, bgalvani, fgiudici, haliu, jwboyer, liali, lrintel, lxin, network-qe, rkhan, sukulkar, thaller, vbenes | ||||||
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
||||||
| Target Release: | 8.0 | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | libteam-1.28-4.el8 | Doc Type: | If docs needed, set a value | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2019-11-05 22:12:52 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Created attachment 1582629 [details]
the same journal for rhel 8.0
Reassigning to NetworkManager for debugging/info/hints. Please see the log in the Description (failing 8.1 nightly) and comment #1 (working 8.0). Hi, I tried on RHEL 8.0 and it breaks when upgrading libteam from 1.27-10 to 1.28-1. The cause of the error is that teamd_config_port_set() fails to find the port by ifname. Xin, do you have any idea why? In NetworkManager we first send the port configuration and then actually enslave the port interface to the team master. Perhaps teamd can't find the port because the interface is not yet enslaved. But this used to work in the past. (In reply to Beniamino Galvani from comment #3) > Hi, > > I tried on RHEL 8.0 and it breaks when upgrading libteam from 1.27-10 to > 1.28-1. The cause of the error is that teamd_config_port_set() fails to find > the port by ifname. > > Xin, do you have any idea why? > > In NetworkManager we first send the port configuration and then actually > enslave the port interface to the team master. Perhaps teamd can't find the > port because the interface is not yet enslaved. But this used to work in the > past. teamd_config_port_set() was introduced on rhel-8.1 by: https://github.com/jpirko/libteam/commit/c8b356a3cd Hangbin, can you help take a look at it? thanks. Reproducer:
# nmcli connection add type team ifname team0 con-name team0+ team.config '{"runner": {"name": "activebackup"}}' ip4 172.20.1.3/24
Connection 'team0+' (b95f23f0-2da4-41bd-969a-df228c8f056e) successfully added.
# nmcli connection add type ethernet ifname enp8s0 master team0 con-name slave+ team-port.config '{"prio": -10, "sticky": true}'
Connection 'slave+' (09221211-9e06-4f75-9b6c-d003f63f0c82) successfully added.
# nmcli connection up slave+
Error: Connection activation failed: Unknown error
# journalctl --since '1 minute ago' | grep team
Jun 26 07:55:10 localhost.localdomain NetworkManager[1021]: <info> [1561550110.9568] device (enp8s0): released from master device team0
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: libteamdctl: cli_usock_process_msg: usock: Error message received: "ConfigUpdateFail"
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: libteamdctl: cli_usock_process_msg: usock: Error message content: "Failed to update config."
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: <error> [1561550168.8452] device (team0): failed to update config for port enp8s0 (err=-22)
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: <info> [1561550168.8457] device (enp8s0): released from master device team0
# rpm -q libteam
libteam-1.28-2.el8.x86_64
# grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux 8.1 Beta (Ootpa)"
Thanks, Beniamino
teamd supports for ports config going first, then ports enslaving. The commit in c4 breaks it, I will make a change in teamd_config_port_update():
tdport = teamd_get_port_by_ifname(ctx, port_name);
if (tdport)
err = teamd_config_port_set(ctx, tdport, port_new_obj);
Hi Vladimir, I noticed that in the NM CI suite there isn't any test setting the 'team-port.config' property of team slaves... could we add at least one to have this feature covered? Thanks! *** Bug 1719110 has been marked as a duplicate of this bug. *** I verified this using reproducer in https://bugzilla.redhat.com/show_bug.cgi?id=1719110#c0 on libteam-1.28-4.el8.x86_64. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3550 |
Created attachment 1582628 [details] journal from the installer environment Description of problem: Installer kickstart tests for team network device configuration are failing in 8.1 (compared to 8.0). Version-Release number of selected component (if applicable): RHEL-8.1.0-20190620.n.0 anaconda 29.19.1.6-1.el8 NetworkManager 1.20.0-0.1.el8 How reproducible: always Steps to Reproduce: Run kickstart installation on VM with 3 network devices (ens3 bridged, ens10, ens11 NATed), booting from ens3, configuring ens10 and ens11 as team slaves by kickstart. kickstart network command: network --device team0 --bootproto dhcp --teamslaves="ens10'{\"prio\": -10, \"sticky\": true}',ens11'{\"prio\": 100}'" --teamconfig="{\"runner\": {\"name\": \"activebackup\"}}" --activate --onboot=no Actual results: The team0 device activation fails, see the attached log, snippet: Jun 20 10:27:32 localhost teamd_team0[2281]: usock: calling method "PortConfigUpdate" Jun 20 10:27:32 localhost NetworkManager[2199]: <debug> [1561026452.0693] platform-linux: do-change-link[3]: success changing link: success Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message received: "ConfigUpdateFail" Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message content: "Failed to update config." Jun 20 10:27:32 localhost teamd_team0[2281]: port_devname "ens10", port_config "{"prio": -10, "sticky": true}" Jun 20 10:27:32 localhost NetworkManager[2199]: <error> [1561026452.0693] device (team0): failed to update config for port ens10 (err=-22) Jun 20 10:27:32 localhost teamd_team0[2281]: ens10: Failed to update config to kernel Jun 20 10:27:32 localhost NetworkManager[2199]: <warn> [1561026452.0693] device (ens10): Activation: connection 'team0 slave 1' could not be enslaved Jun 20 10:27:32 localhost teamd_team0[2281]: Failed to update config for port "ens10". Expected results: The team0 device is activated as expected. Additional info: I am not quite sure if our kickstart port configuration is valid but it works on 8.0. We don't have any record of the test being successful so far in 8.1.