Bug 1211758

Summary: The VF is failed to be used in macvtap-passthrough network if it has been used in hostdev network
Product: Red Hat Enterprise Linux 7 Reporter: Laine Stump <laine>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.2CC: dyuan, honzhang, jdenemar, laine, mzhan, rbalakri, virt-bugs, vyasevic
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.2.17-4.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1113474 Environment:
Last Closed: 2015-11-19 06:28:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Laine Stump 2015-04-14 19:38:59 UTC
+++ This bug was initially created as a clone of Bug #1113474 +++

Description of problem:
The VF is failed to be used in macvtap-passthrough network if it has been used in hostdev network

See original bug for all the gory details.

The fix is to use netlink RTM_SETLINK for the PF+VF# to set the VF's MAC address in all cases (previously we were using ioctl(SIOCSIFHWADDR) when doing macvtap passthrough).


--- Additional comment from Laine Stump on 2015-04-10 15:03:24 EDT ---

I've tried modifying libvirt to always use netlink STM_SETLINK (sent for the PF, with vf# as a parameter) whenever possible:

 https://www.redhat.com/archives/libvir-list/2015-April/msg00422.html

and this appears to solve the problem of setting the MAC address.

There is an additional problem though - when the VF is re-attached to the host driver after hostdev passthrough is finished, it of course becomes ~IFF_UP. So we can successfully setup macvtap passthrough, but no traffic will pass until the VF is set IFF_UP. libvirt should be doing this automatically.

--- Additional comment from Laine Stump on 2015-04-14 15:03:01 EDT ---

The following patch posted upstream solves the problem with IFF_UP:

https://www.redhat.com/archives/libvir-list/2015-April/msg00510.html


Both patches are yet to be ACKed (or pushed, of course :-)

Comment 3 Laine Stump 2015-08-25 03:42:43 UTC
I should also point out that, on RHEL6 at least, this patch causes a problem with the Cisco enic driver, which hasn't implemented the driver function that's behind IFLA_VFINFO_LIST. If the enic driver in 7.2 is also missing this function, we will need to implement some sort of workaround.

Comment 5 Laine Stump 2015-08-26 05:18:42 UTC
I filed Bug 1257004 against RHEL7 to remedy the problem in Comment 3 - it turns out that although the enic driver is missing the function that provides IFLA_VFINFO_LIST, libvirt doesn't really need it.

Comment 6 hongming 2015-09-08 07:05:41 UTC
Verify it as follows. The result is expected. Move its status to VERIFIED.

# uname -r
3.10.0-306.0.1.el7.x86_64
# rpm -q libvirt
libvirt-1.2.17-7.el7.x86_64


# lspci|grep 82576
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)


# virsh nodedev-list --tree
computer
  |
.....

  |   +- pci_0000_03_10_0
  |   |   |
  |   |   +- net_enp3s16_02_c4_9d_51_07_c6

.....


# virsh net-list
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 hostdev              active     no            yes
 passthrough          active     no            yes

# virsh net-dumpxml hostdev
<network>
  <name>hostdev</name>
  <uuid>e1c26bc3-5c86-42f6-a562-d860088283b6</uuid>
  <forward mode='hostdev' managed='yes'>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
  </forward>
</network>

# virsh net-dumpxml passthrough
<network>
  <name>passthrough</name>
  <uuid>d65534ab-5de1-4071-a21f-0133b308fff3</uuid>
  <forward dev='enp3s16' mode='passthrough'>
    <interface dev='enp3s16'/>
  </forward>
</network>

# cat if-hostdev.xml
<interface type='network'>
    <source network='hostdev'/>
</interface>

# virsh attach-device r7-1 if-hostdev.xml
Device attached successfully

# virsh net-dumpxml hostdev
<network connections='1'>
  <name>hostdev</name>
  <uuid>e1c26bc3-5c86-42f6-a562-d860088283b6</uuid>
  <forward mode='hostdev' managed='yes'>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
  </forward>
</network>

# virsh destroy r7-1
Domain r7-1 destroyed

# virsh start r7-1
Domain r7-1 started

# cat if-passthrough.xml
<interface type='network'>
    <source network='passthrough'/>
</interface>

# virsh attach-device r7-1 if-passthrough.xml
Device attached successfully

# virsh dumpxml r7-1|grep "interface type='direct'" -A7
    <interface type='direct'>
      <mac address='52:54:00:ef:f9:be'/>
      <source network='passthrough' dev='enp3s16' mode='passthrough'/>
      <target dev='macvtap0'/>
      <model type='rtl8139'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </interface>

#  ip -d link show macvtap0
16: macvtap0@enp3s16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT qlen 500
    link/ether 02:c4:9d:51:07:c6 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    macvtap  mode passthru addrgenmode eui64

Comment 8 errata-xmlrpc 2015-11-19 06:28:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html