Bug 1480231

Summary: link state element not honored for interface hostdevs
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: yalzhang <yalzhang>
Component: libvirtAssignee: Virtualization Maintenance <virt-maint>
Status: CLOSED DEFERRED QA Contact: jiyan <jiyan>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.0CC: chhu, laine, rbalakri, xuzhang, yafu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-02-18 13:31:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1558351    

Description yalzhang@redhat.com 2017-08-10 12:56:57 UTC
Description of problem:
hotplug a hostdev interface with <link state = down/> will succeed,but the link state make no sense in hostdev interface type.

Version-Release number of selected component (if applicable):
# rpm -q libvirt qemu-kvm-rhev
libvirt-3.2.0-14.el7_4.2.x86_64
qemu-kvm-rhev-2.9.0-16.el7_4.3.x86_64

How reproducible:
100%

Steps to Reproduce:
1. prepare a hostdev interface xml with <link state='down'/>
# virsh start rhel
Domain rhel started

# cat vf.xml
<interface type='hostdev' managed='yes'>
  <source>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
  </source>
<link state='down'/>
<target dev='test'/>
</interface>

2. hotplug to a guest
# virsh attach-device rhel vf.xml
Device attached successfully

# virsh dumpxml rhel | grep /interface -B10
    <interface type='hostdev' managed='yes'>
      <mac address='52:54:00:10:a6:15'/>
      <driver name='vfio'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x10'
function='0x0'/>
      </source>
      <target dev='test'/>
      <link state='down'/>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
    </interface>

3. login the guest, the interface can work smoothly, not down
# ethtool ens3
Settings for ens3:
...
Link detected: yes
# ping www.baidu.com
PING www.a.shifen.com (119.75.213.51) 56(84) bytes of data.
64 bytes from 119.75.213.51 (119.75.213.51): icmp_seq=1 ttl=49 time=2.88 ms
....

4. on the host
# virsh domiflist rhel
Interface  Type       Source     Model       MAC
-------------------------------------------------------
test       hostdev    -          -           52:54:00:cb:5d:f6

# virsh domif-getlink rhel test
test down


# virsh domif-setlink  rhel test up
error: Failed to update interface link state
error: Operation not supported: cannot change config of 'hostdev' network type

# virsh domifstat rhel test
error: Failed to get interface stats rhel test
error: internal error: /proc/net/dev: Interface not found

5.  try to start a guest with same setting, it failed to start as couldn't set link state as down
# virsh destroy rhel
vDomain rhel destroyed

# virsh attach-device rhel vf.xml --config
Device attached successfully

# virsh dumpxml rhel | grep /interface -B7
    <interface type='hostdev' managed='yes'>
      <mac address='52:54:00:05:88:04'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x10'
function='0x0'/>
      </source>
      <link state='down'/>
     <target dev='test'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
    </interface>

# virsh start rhel
error: Failed to start domain rhel
error: operation failed: Couldn't set link state on interface: hostdev0

Actual results:
libvirt do not check the link state setting during hotplug hostdev interface

Expected results:
libvirt should check the link state during hotplug, or hostdev type
interface should reject link state setting.

Additional info:

Comment 2 yalzhang@redhat.com 2017-08-23 07:31:15 UTC
one more scenario: 
update link state persistent config of vhostdev interface type, libvirtd will crash

# rpm -q libvirt 
libvirt-3.2.0-14.el7_4.3.x86_64

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     rhel7.4                        shut off

# virsh dumpxml rhel7.4 | grep hostdev -A6
    <interface type='hostdev' managed='yes'>
      <mac address='52:54:00:4a:de:7e'/>
      <source>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x6'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>

# virsh domif-getlink rhel7.4 52:54:00:4a:de:7e
52:54:00:4a:de:7e up

# pidof libvirtd
14954

# virsh domif-setlink rhel7.4 52:54:00:4a:de:7e up
error: Disconnected from qemu:///system due to I/O error
error: Failed to update interface link state
error: End of file while reading data: Input/output error

# pidof libvirtd
15203

or when guest is running, setlink with --config will trigger the failure as well
 
# virsh domif-setlink rhel7.4 52:54:00:4a:de:7e down --config
error: Disconnected from qemu:///system due to I/O error
error: Failed to update interface link state
error: End of file while reading data: Input/output error

Comment 3 Laine Stump 2017-08-30 21:49:43 UTC
The ability for the host to control the link state of an assigned VF directly is handled by setting the IFLA_VF_LINK_STATE attribute in an RTM_SETLINK netlink message to the PF. However, IFLA_VF_LINK_STATE is supported by only some SRIOV drivers (upstream in the 3.x days it was only the mlx4 driver that supported it; now it looks like mlx*, i40e, benet, bnx*, qeg, and sfc support it. igb and ixgbe don't).

This means we can add support for this, and it will work for some sriov cards, but not all. If the attribute is set for a VF on a card that doesn't support it, this will result in an error.

Comment 4 Laine Stump 2020-02-11 03:11:08 UTC
The crash mentioned in the Bug summary is almost certainly the one that was described in Bug 1558934 and fixed upstream in commit 7e490cdad. This was in upstream libvirt-5.7.0

Beyond that, there is a validation problem here, where we need to disallow setting link state for hostdev interfaces, or alternately we can attempt to support setting link state for these interfaces (which may not work for all hardware, as described in Comment 3).

Comment 5 Jaroslav Suchanek 2020-02-18 13:31:40 UTC
This bug was closed deferred as a result of bug triage.

Please reopen if you disagree and provide justification why this bug should
get enough priority. Most important would be information about impact on
customer or layered product. Please indicate requested target release.