RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1722449 - Anaconda network team configuration tests failing with rhel 8.1 nightlies ("failed to update config for port")
Summary: Anaconda network team configuration tests failing with rhel 8.1 nightlies ("f...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: libteam
Version: 8.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: 8.0
Assignee: Xin Long
QA Contact: LiLiang
URL:
Whiteboard:
: 1719110 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-06-20 11:43 UTC by Radek Vykydal
Modified: 2021-09-03 15:14 UTC (History)
13 users (show)

Fixed In Version: libteam-1.28-4.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-05 22:12:52 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
journal from the installer environment (772.94 KB, text/plain)
2019-06-20 11:43 UTC, Radek Vykydal
no flags Details
the same journal for rhel 8.0 (692.62 KB, text/plain)
2019-06-20 11:44 UTC, Radek Vykydal
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:3550 0 None None None 2019-11-05 22:13:07 UTC

Description Radek Vykydal 2019-06-20 11:43:57 UTC
Created attachment 1582628 [details]
journal from the installer environment

Description of problem:

Installer kickstart tests for team network device configuration are failing in 8.1 (compared to 8.0).

Version-Release number of selected component (if applicable):

RHEL-8.1.0-20190620.n.0
anaconda 29.19.1.6-1.el8
NetworkManager 1.20.0-0.1.el8

How reproducible:

always

Steps to Reproduce:

Run kickstart installation on VM with 3 network devices (ens3 bridged, ens10, ens11 NATed), booting from ens3, configuring ens10 and ens11 as team slaves by kickstart.

kickstart network command:

network --device team0 --bootproto dhcp --teamslaves="ens10'{\"prio\": -10, \"sticky\": true}',ens11'{\"prio\": 100}'" --teamconfig="{\"runner\": {\"name\": \"activebackup\"}}" --activate --onboot=no

Actual results:

The team0 device activation fails, see the attached log, snippet:

Jun 20 10:27:32 localhost teamd_team0[2281]: usock: calling method "PortConfigUpdate"
  Jun 20 10:27:32 localhost NetworkManager[2199]: <debug> [1561026452.0693] platform-linux: do-change-link[3]: success changing link: success
  Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message received: "ConfigUpdateFail"
  Jun 20 10:27:32 localhost NetworkManager[2199]: libteamdctl: cli_usock_process_msg: usock: Error message content: "Failed to update config."
  Jun 20 10:27:32 localhost teamd_team0[2281]: port_devname "ens10", port_config "{"prio": -10, "sticky": true}"
  Jun 20 10:27:32 localhost NetworkManager[2199]: <error> [1561026452.0693] device (team0): failed to update config for port ens10 (err=-22)
  Jun 20 10:27:32 localhost teamd_team0[2281]: ens10: Failed to update config to kernel
  Jun 20 10:27:32 localhost NetworkManager[2199]: <warn>  [1561026452.0693] device (ens10): Activation: connection 'team0 slave 1' could not be enslaved
  Jun 20 10:27:32 localhost teamd_team0[2281]: Failed to update config for port "ens10".


Expected results:

The team0 device is activated as expected.

Additional info:

I am not quite sure if our kickstart port configuration is valid but it works on 8.0.

We don't have any record of the test being successful so far in 8.1.

Comment 1 Radek Vykydal 2019-06-20 11:44:59 UTC
Created attachment 1582629 [details]
the same journal for rhel 8.0

Comment 2 Radek Vykydal 2019-06-20 11:48:04 UTC
Reassigning to NetworkManager for debugging/info/hints. Please see the log in the Description (failing 8.1 nightly) and comment #1 (working 8.0).

Comment 3 Beniamino Galvani 2019-06-21 14:45:09 UTC
Hi,

I tried on RHEL 8.0 and it breaks when upgrading libteam from 1.27-10 to 1.28-1. The cause of the error is that teamd_config_port_set() fails to find the port by ifname.

Xin, do you have any idea why?

In NetworkManager we first send the port configuration and then actually enslave the port interface to the team master. Perhaps teamd can't find the port because the interface is not yet enslaved. But this used to work in the past.

Comment 4 Xin Long 2019-06-24 04:26:38 UTC
(In reply to Beniamino Galvani from comment #3)
> Hi,
> 
> I tried on RHEL 8.0 and it breaks when upgrading libteam from 1.27-10 to
> 1.28-1. The cause of the error is that teamd_config_port_set() fails to find
> the port by ifname.
> 
> Xin, do you have any idea why?
> 
> In NetworkManager we first send the port configuration and then actually
> enslave the port interface to the team master. Perhaps teamd can't find the
> port because the interface is not yet enslaved. But this used to work in the
> past.

teamd_config_port_set() was introduced on rhel-8.1 by:

  https://github.com/jpirko/libteam/commit/c8b356a3cd

Hangbin, can you help take a look at it? thanks.

Comment 5 Beniamino Galvani 2019-06-26 12:00:08 UTC
Reproducer:

# nmcli connection add type team ifname team0 con-name team0+ team.config '{"runner": {"name": "activebackup"}}' ip4 172.20.1.3/24
Connection 'team0+' (b95f23f0-2da4-41bd-969a-df228c8f056e) successfully added.

# nmcli connection add type ethernet ifname enp8s0 master team0 con-name slave+ team-port.config '{"prio": -10, "sticky": true}'
Connection 'slave+' (09221211-9e06-4f75-9b6c-d003f63f0c82) successfully added.

# nmcli connection up slave+
Error: Connection activation failed: Unknown error

# journalctl --since '1 minute ago' | grep team
Jun 26 07:55:10 localhost.localdomain NetworkManager[1021]: <info>  [1561550110.9568] device (enp8s0): released from master device team0
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: libteamdctl: cli_usock_process_msg: usock: Error message received: "ConfigUpdateFail"
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: libteamdctl: cli_usock_process_msg: usock: Error message content: "Failed to update config."
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: <error> [1561550168.8452] device (team0): failed to update config for port enp8s0 (err=-22)
Jun 26 07:56:08 localhost.localdomain NetworkManager[1021]: <info>  [1561550168.8457] device (enp8s0): released from master device team0

# rpm -q libteam
libteam-1.28-2.el8.x86_64

# grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Red Hat Enterprise Linux 8.1 Beta (Ootpa)"

Comment 6 Xin Long 2019-06-26 16:36:23 UTC
Thanks, Beniamino

teamd supports for ports config going first, then ports enslaving. The commit in c4 breaks it, I will make a change in teamd_config_port_update():

        tdport = teamd_get_port_by_ifname(ctx, port_name);
        if (tdport)
                err = teamd_config_port_set(ctx, tdport, port_new_obj);

Comment 7 Beniamino Galvani 2019-06-28 08:54:27 UTC
Hi Vladimir,

I noticed that in the NM CI suite there isn't any test setting the 'team-port.config' property of team slaves... could we add at least one to have this feature covered? Thanks!

Comment 13 Hangbin Liu 2019-07-15 09:03:27 UTC
*** Bug 1719110 has been marked as a duplicate of this bug. ***

Comment 14 LiLiang 2019-07-19 03:03:54 UTC
I verified this using reproducer in https://bugzilla.redhat.com/show_bug.cgi?id=1719110#c0 on libteam-1.28-4.el8.x86_64.

Comment 17 errata-xmlrpc 2019-11-05 22:12:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3550


Note You need to log in before you can comment on or make changes to this bug.