Bug 2003883 - Failed to update the VFs configuration of network interface card type 82599ES and X520
Summary: Failed to update the VFs configuration of network interface card type 82599ES...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.4.8
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ovirt-4.5.0
: 4.5.0
Assignee: Ales Musil
QA Contact: Michael Burman
URL:
Whiteboard:
Depends On: 2021409 2024351
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-09-14 05:11 UTC by zhoujunqin
Modified: 2022-05-26 16:23 UTC (History)
6 users (show)

Fixed In Version: ovirt-engine-4.5.0
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-26 16:23:07 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
/var/log/vdsm/vdsm.log (29.47 KB, text/plain)
2021-09-14 05:11 UTC, zhoujunqin
no flags Details
supervdsm-bug2003883.log (136.05 KB, text/plain)
2021-10-27 09:27 UTC, zhoujunqin
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-43638 0 None None None 2021-09-14 05:12:47 UTC
Red Hat Product Errata RHSA-2022:4711 0 None None None 2022-05-26 16:23:16 UTC

Description zhoujunqin 2021-09-14 05:11:53 UTC
Created attachment 1822920 [details]
/var/log/vdsm/vdsm.log

Description of problem:
Failed to update the VFs configuration of network interface card type 82599ES and X520

Message as following:
VDSM host_hhan command HostDevChangeNumVfsVDS failed: Internal JSON-RPC error: {'reason': '\ndesired\n=======\n---\nname: ev0\ntype: ethernet\nstate: down\n\ncurrent\n=======\n--- {}\n\ndifference\n==========\n--- desired\n+++ current\n@@ -1,4 +1 @@\n----\n-name: ev0\n-type: ethernet\n-state: down\n+--- {}\n\n'}

Version-Release number of selected component (if applicable):
RHV ENGINE INFO:
ovirt-engine-4.4.8.5-0.4.el8ev.noarch

RHEL HOST INFO:
vdsm-4.40.80.6-1.el8ev.x86_64
libvirt-7.6.0-3.module+el8.5.0+12510+80564ecf.x86_64
qemu-kvm-6.0.0-29.module+el8.5.0+12386+43574bac.x86_64
kernel-4.18.0-340.el8.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare a registerted rhv node with SRIOV support
Tab Compute -> Hosts -> Edit(eg:host-a) -> Kernel : choose Hostdev Passthrough & SR-IOV(and add intel_iommu=on or amd_iommu=on in 'Kernel command line'); save configurations.
Management -> Maintenance host; Installation -> reinstall, then active the sriov host again.

The host ethernet info - 82:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

2. Tab Networks :
New -> new a logical network	

3. Change network profile to passthrough
Tab Networks :
Network->vNIC Profiles (Select newly added ‘sriov-test’ network)->Edit
->choose Passthrough.

Then save configurations.
	
4. Tab Compute ->Hosts - > Network interface -> Setup Host Networks
Choose the interface with SR-IOV tag; edit
Click Number of VF Setting: set the number
Drag ( eg: testa) to a PF on the left part, then save the configuration.


Actual results:
Failed to save the configuration with the error message:
"""
Operation Canceled
Error while executing at least some of the action tasks:

enp130s0f0:
Failed to change the number of virtual functions.
"""

Expected results:
Fix it.

Additional info:
1. Get from "/var/log/messages" log got from the registered host
Sep 14 01:03:42 dell-per730-36 vdsm[13551]: ERROR Internal server error#012Traceback (most recent call last):#012  File "/usr/lib/python3.6/site-packages/yajsonrpc/__init__.py", line 349, in _handle_request#012    res = method(**params)#012  File "/usr/lib/python3.6/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod#012    result = fn(*methodArgs)#012  File "<decorator-gen-467>", line 2, in hostdevChangeNumvfs#012  File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method#012    ret = func(*args, **kwargs)#012  File "/usr/lib/python3.6/site-packages/vdsm/API.py", line 1439, in hostdevChangeNumvfs#012    hostdev.change_numvfs(deviceName, numvfs)#012  File "/usr/lib/python3.6/site-packages/vdsm/common/hostdev.py", line 818, in change_numvfs#012    supervdsm.getProxy().change_numvfs(numvfs, net_name)#012  File "/usr/lib/python3.6/site-packages/vdsm/common/supervdsm.py", line 56, in __call__#012    return callMethod()#012  File "/usr/lib/python3.6/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda>#012    **kwargs)#012  File "<string>", line 2, in change_numvfs#012  File "/usr/lib64/python3.6/multiprocessing/managers.py", line 772, in _callmethod#012    raise convert_to_error(kind, result)#012libnmstate.error.NmstateVerificationError: #012desired#012=======#012---#012name: ev0#012type: ethernet#012state: down#012#012current#012=======#012--- {}#012#012difference#012==========#012--- desired#012+++ current#012@@ -1,4 +1 @@#012----#012-name: ev0#012-type: ethernet#012-state: down#012+--- {}

2. I will attach the vdsm.log got from the registered host.

3. I can also reproduce this issue with SRIOV card type - "Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)"

4. I can't reproduce this issue with SRIOV card type - "Ethernet controller: Mellanox Technologies MT2892 Family [ConnectX-6 Dx]".

Comment 1 Sharon Gratch 2021-10-21 12:45:05 UTC
This seems like a networking issue, not related to UX

Comment 2 Ales Musil 2021-10-27 08:37:52 UTC
Hi, can you please also provide relevant part from supervdsm.log?

Thanks, 
Ales

Comment 3 zhoujunqin 2021-10-27 09:25:21 UTC
Created attachment 1837542 [details]
Re-attach the vdsm-bug2003883.log

Comment 4 zhoujunqin 2021-10-27 09:27:36 UTC
Created attachment 1837543 [details]
supervdsm-bug2003883.log

Comment 5 zhoujunqin 2021-10-27 09:28:39 UTC
(In reply to Ales Musil from comment #2)
> Hi, can you please also provide relevant part from supervdsm.log?
> 
> Thanks, 
> Ales

Done, Ales, please recheck the attachments, thanks.

BR,
juzhou.

Comment 6 Ales Musil 2021-11-02 08:35:35 UTC
Hi, 

can we borrow the host to test it out?

Thanks,
Ales

Comment 8 Gris Ge 2021-11-09 08:00:57 UTC
Confirmed as nmstate bug. Please track the efforts in https://bugzilla.redhat.com/show_bug.cgi?id=2021409

Comment 14 Michael Burman 2022-03-22 08:07:03 UTC
Verified on - rhvm-4.5.0-0.237.el8ev.noarch with:

nmstate-1.2.1-1.el8.x86_64
vdsm-4.50.0.10-1.el8ev.x86_64
NetworkManager-1.36.0-3.el8.x86_64

Comment 15 Michael Burman 2022-03-22 08:10:59 UTC
Tested on:

19:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
        Subsystem: Dell Ethernet 10G 4P X520/I350 rNDC
        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
19:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
        Subsystem: Dell Ethernet 10G 4P X520/I350 rNDC
        Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
19:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

Comment 20 errata-xmlrpc 2022-05-26 16:23:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: RHV Manager (ovirt-engine) [ovirt-4.5.0] security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:4711


Note You need to log in before you can comment on or make changes to this bug.