Bug 2208025

Summary:	MTU change to 1500 after reboot for interfaces used for vf with BCM57840
Product:	Red Hat OpenStack	Reporter:	Jean-Francois Beaudoin <jbeaudoi>
Component:	os-net-config	Assignee:	OSP Team <rhos-maint>
Status:	NEW ---	QA Contact:	Eran Kuris <ekuris>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	16.2 (Train)	CC:	bfournie, chrisw, gurpsing, hakhande, hbrock, jmarti, jslagle, ksundara, mburns, mlavalle, rjarry, scohen, ykarel
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jean-Francois Beaudoin 2023-05-17 17:26:04 UTC

Description of problem:
After reboot of the compute node, MTU reset to 1500 on interfaces used for virtual function, for SRIOV.
Only happens with BCM57840 NetXtreme II card.

Version-Release number of selected component (if applicable):
Red Hat OpenStack Platform release 16.2.4 (Train)
Red Hat Enterprise Linux release 8.4 (Ootpa)
puppet-neutron-15.6.0-2.20220910230134.b2d4066.el8ost.noarch
python3-neutronclient-6.14.1-2.20220111010835.a09e824.el8ost.noarch


How reproducible:
Reboot compute node with this card.

Steps to Reproduce:
1.reboot compute node
2.
3.

Actual results:
MTU switch from 9000 to 1500 after reboot for theses interfaces.

Expected results:
MTU stays the same after reboot.

Additional info:
It can be restored manually after reboot with:
echo 0 > /sys/class/net/ens3f2/device/sriov_numvfs
ip link set dev ens3f2 mtu 9000
echo 10 > /sys/class/net/ens3f2/device/sriov_numvfs

Comment 2 Karthik Sundaravel 2023-05-23 02:34:48 UTC

What surprises me is the presence and trigger of the script '/etc/sysconfig/allocate_vfs ens3f3'. This script is generated by puppet in older OSPs and will be removed by os-net-config during upgrade. [1] 
During reboot, is the numvfs configured by multiple scripts ? is numvfs successfully set after reboot ?
Can you please upload the sos-report in the portal ?

[1] https://github.com/openstack/os-net-config/blob/stable/train/os_net_config/sriov_config.py#L222

Comment 7 Karthik Sundaravel 2023-05-31 12:44:07 UTC

We have 3 scripts that are trying to configure the numvfs, udev based rules failed to set the numvfs. But however the sriov_config service effectively set the numvfs. So in a way this is fine.
But after that the setting of MTU fails with NetworkManager.

The workaround suggests that it could be a limitation with BCM57840 drivers that MTU could not be configured after the creation of VFs.
Can this be looked in from BCM57840  driver perspective ?