Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Created attachment 1582628[details]
journal from the installer environment
Description of problem:
Installer kickstart tests for team network device configuration are failing in 8.1 (compared to 8.0).
Version-Release number of selected component (if applicable):
RHEL-8.1.0-20190620.n.0
anaconda 29.19.1.6-1.el8
NetworkManager 1.20.0-0.1.el8
How reproducible:
always
Steps to Reproduce:
Run kickstart installation on VM with 3 network devices (ens3 bridged, ens10, ens11 NATed), booting from ens3, configuring ens10 and ens11 as team slaves by kickstart.
kickstart network command:
network --device team0 --bootproto dhcp --teamslaves="ens10'{\"prio\": -10, \"sticky\": true}',ens11'{\"prio\": 100}'" --teamconfig="{\"runner\": {\"name\": \"activebackup\"}}" --activate --onboot=no
Actual results:
The team0 device activation fails, see the attached log, snippet:
Jun 20 10:27:32 localhost teamd_team0[2281]: usock: calling method "PortConfigUpdate"
Jun 20 10:27:32 localhost NetworkManager[2199]: <debug> [1561026452.0693] platform-linux: do-change-link[3]: success changing link: success
Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message received: "ConfigUpdateFail"
Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message content: "Failed to update config."
Jun 20 10:27:32 localhost teamd_team0[2281]: port_devname "ens10", port_config "{"prio": -10, "sticky": true}"
Jun 20 10:27:32 localhost NetworkManager[2199]: <error> [1561026452.0693] device (team0): failed to update config for port ens10 (err=-22)
Jun 20 10:27:32 localhost teamd_team0[2281]: ens10: Failed to update config to kernel
Jun 20 10:27:32 localhost NetworkManager[2199]: <warn> [1561026452.0693] device (ens10): Activation: connection 'team0 slave 1' could not be enslaved
Jun 20 10:27:32 localhost teamd_team0[2281]: Failed to update config for port "ens10".
Expected results:
The team0 device is activated as expected.
Additional info:
I am not quite sure if our kickstart port configuration is valid but it works on 8.0.
We don't have any record of the test being successful so far in 8.1.
Reassigning to NetworkManager for debugging/info/hints. Please see the log in the Description (failing 8.1 nightly) and comment #1 (working 8.0).
Comment 3Beniamino Galvani
2019-06-21 14:45:09 UTC
Hi,
I tried on RHEL 8.0 and it breaks when upgrading libteam from 1.27-10 to 1.28-1. The cause of the error is that teamd_config_port_set() fails to find the port by ifname.
Xin, do you have any idea why?
In NetworkManager we first send the port configuration and then actually enslave the port interface to the team master. Perhaps teamd can't find the port because the interface is not yet enslaved. But this used to work in the past.
(In reply to Beniamino Galvani from comment #3)
> Hi,
>
> I tried on RHEL 8.0 and it breaks when upgrading libteam from 1.27-10 to
> 1.28-1. The cause of the error is that teamd_config_port_set() fails to find
> the port by ifname.
>
> Xin, do you have any idea why?
>
> In NetworkManager we first send the port configuration and then actually
> enslave the port interface to the team master. Perhaps teamd can't find the
> port because the interface is not yet enslaved. But this used to work in the
> past.
teamd_config_port_set() was introduced on rhel-8.1 by:
https://github.com/jpirko/libteam/commit/c8b356a3cd
Hangbin, can you help take a look at it? thanks.
Comment 5Beniamino Galvani
2019-06-26 12:00:08 UTC
Reproducer:
# nmcli connection add type team ifname team0 con-name team0+ team.config '{"runner": {"name": "activebackup"}}' ip4 172.20.1.3/24
Connection 'team0+' (b95f23f0-2da4-41bd-969a-df228c8f056e) successfully added.
# nmcli connection add type ethernet ifname enp8s0 master team0 con-name slave+ team-port.config '{"prio": -10, "sticky": true}'
Connection 'slave+' (09221211-9e06-4f75-9b6c-d003f63f0c82) successfully added.
# nmcli connection up slave+
Error: Connection activation failed: Unknown error
# journalctl --since '1 minute ago' | grep team
Jun 26 07:55:10 localhost.localdomain NetworkManager[1021]: <info> [1561550110.9568] device (enp8s0): released from master device team0
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: libteamdctl: cli_usock_process_msg: usock: Error message received: "ConfigUpdateFail"
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: libteamdctl: cli_usock_process_msg: usock: Error message content: "Failed to update config."
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: <error> [1561550168.8452] device (team0): failed to update config for port enp8s0 (err=-22)
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: <info> [1561550168.8457] device (enp8s0): released from master device team0
# rpm -q libteam
libteam-1.28-2.el8.x86_64
# grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux 8.1 Beta (Ootpa)"
Thanks, Beniamino
teamd supports for ports config going first, then ports enslaving. The commit in c4 breaks it, I will make a change in teamd_config_port_update():
tdport = teamd_get_port_by_ifname(ctx, port_name);
if (tdport)
err = teamd_config_port_set(ctx, tdport, port_new_obj);
Comment 7Beniamino Galvani
2019-06-28 08:54:27 UTC
Hi Vladimir,
I noticed that in the NM CI suite there isn't any test setting the 'team-port.config' property of team slaves... could we add at least one to have this feature covered? Thanks!
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2019:3550
Created attachment 1582628 [details] journal from the installer environment Description of problem: Installer kickstart tests for team network device configuration are failing in 8.1 (compared to 8.0). Version-Release number of selected component (if applicable): RHEL-8.1.0-20190620.n.0 anaconda 29.19.1.6-1.el8 NetworkManager 1.20.0-0.1.el8 How reproducible: always Steps to Reproduce: Run kickstart installation on VM with 3 network devices (ens3 bridged, ens10, ens11 NATed), booting from ens3, configuring ens10 and ens11 as team slaves by kickstart. kickstart network command: network --device team0 --bootproto dhcp --teamslaves="ens10'{\"prio\": -10, \"sticky\": true}',ens11'{\"prio\": 100}'" --teamconfig="{\"runner\": {\"name\": \"activebackup\"}}" --activate --onboot=no Actual results: The team0 device activation fails, see the attached log, snippet: Jun 20 10:27:32 localhost teamd_team0[2281]: usock: calling method "PortConfigUpdate" Jun 20 10:27:32 localhost NetworkManager[2199]: <debug> [1561026452.0693] platform-linux: do-change-link[3]: success changing link: success Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message received: "ConfigUpdateFail" Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message content: "Failed to update config." Jun 20 10:27:32 localhost teamd_team0[2281]: port_devname "ens10", port_config "{"prio": -10, "sticky": true}" Jun 20 10:27:32 localhost NetworkManager[2199]: <error> [1561026452.0693] device (team0): failed to update config for port ens10 (err=-22) Jun 20 10:27:32 localhost teamd_team0[2281]: ens10: Failed to update config to kernel Jun 20 10:27:32 localhost NetworkManager[2199]: <warn> [1561026452.0693] device (ens10): Activation: connection 'team0 slave 1' could not be enslaved Jun 20 10:27:32 localhost teamd_team0[2281]: Failed to update config for port "ens10". Expected results: The team0 device is activated as expected. Additional info: I am not quite sure if our kickstart port configuration is valid but it works on 8.0. We don't have any record of the test being successful so far in 8.1.