Bug 1731391

Summary: Trust on does not take effect when vf add to ovs bridge
Product: Red Hat Enterprise Linux Fast Datapath Reporter: liting <tli>
Component: DPDKAssignee: Maxime Coquelin <maxime.coquelin>
DPDK sub component: sriov QA Contact: liting <tli>
Status: NEW --- Docs Contact:
Severity: medium    
Priority: unspecified CC: ctrautma, jhsiao, maxime.coquelin
Version: FDP 19.D   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description liting 2019-07-19 10:10:26 UTC
Description of problem:
Trust on does not take effect when vf add to ovs bridge

Version-Release number of selected component (if applicable):
[root@dell-per730-15 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.0 (Ootpa)
[root@dell-per730-15 ~]# uname -a
Linux dell-per730-15.rhts.eng.pek2.redhat.com 4.18.0-80.el8.x86_64 #1 SMP Wed Mar 13 12:02:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@dell-per730-15 ~]# rpm -qa|grep openv
python3-openvswitch2.11-2.11.0-12.el8fdp.x86_64
openvswitch2.11-2.11.0-12.el8fdp.x86_64
openvswitch-selinux-extra-policy-1.0-12.el8fdp.noarch
openvswitch2.11-test-2.11.0-12.el8fdp.noarch
[root@dell-per730-15 ~]# rpm -qa|grep dpdk
dpdk-18.11-8.el8.x86_64
dpdk-tools-18.11-8.el8.x86_64

[root@dell-per730-15 ~]# ethtool -i enp5s0f0
driver: ixgbe
version: 5.1.0-k
firmware-version: 0x000161ae
expansion-rom-version: 
bus-info: 0000:05:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

How reproducible:


Steps to Reproduce:
Dell15 ixgbe nic connect with Dell02 ixgbe nic directly.
On Dell02 system: 
1. Create 1 vf for pf enp131s0f0
echo 1 > /sys/devices/pci0000:80/0000:80:03.0/0000:83:00.0/sriov_numvfs
And check the two vfs as following.
 enp131s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 90:e2:ba:90:d9:34 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC d2:2d:b9:fc:fc:e5, spoof checking on, link-state auto, trust off, query_rss off

2. Bind vf 1 to dpdk, and add dpdk0 to ovs bridge.
 /usr/share/dpdk/usertools/dpdk-devbind.py -b vfio-pci 0000:83:10.2
 systemctl restart openvswitch
 ovs-vsctl set Open_vSwitch . 'other_config={}'
 ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
 ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=1024,1024
 ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0xa000a000
 ovs-vsctl add-br ovsbr0 -- set bridge ovsbr0 datapath_type=netdev
 ovs-vsctl add-port ovsbr0 dpdk0 -- set Interface dpdk0 type=dpdk type=dpdk options:dpdk-devargs=0000:83:10.0
 ovs-vsctl add-port ovsbr0 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient -- set Interface dpdkvhostuserclient0 options:vhost-server-path=/tmp/dpdkvhostuserclient0

3. inside guest, configure mac
ip link set addr 00:00:00:00:00:01 dev eth0
4. Configure vf 0 trust on
ip link set enp131s0f0 vf 0 trust on
5. inside guest, configure mac
ip link set addr 00:00:00:00:00:01 dev eth0

Actual results:
Step3, configure successfully.
Step5, configure successfully.

Expected results:
Step3 configure failed when trust is off.


Additional info:
guest xml as following.
[root@dell-per730-02 ~]# virsh dumpxml master3
<domain type='kvm' id='1'>
  <name>master3</name>
  <uuid>37425e76-af6a-44a6-aba0-73434afe34c0</uuid>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>5242880</currentMemory>
  <memoryBacking>
    <hugepages>
      <page size='1048576' unit='KiB'/>
    </hugepages>
    <access mode='shared'/>
  </memoryBacking>
  <vcpu placement='static'>3</vcpu>
  <cputune>
    <emulatorpin cpuset='3'/>
  </cputune>
  <numatune>
    <memory mode='strict' nodeset='1'/>
  </numatune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <feature policy='require' name='tsc-deadline'/>
    <numa>
      <cell id='0' cpus='0-2' memory='8388608' unit='KiB' memAccess='shared'/>
    </numa>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/master3.qcow2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='vhostuser'>
      <mac address='d2:2d:b9:fc:fc:e5'/>
      <source type='unix' path='/tmp/dpdkvhostuserclient0' mode='server'/>
      <target dev='dpdkvhostuserclient0'/>
      <model type='virtio'/>
      <driver name='vhost'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-1-master3/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c119,c145</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c119,c145</imagelabel>
  </seclabel>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+107:+1001</label>
    <imagelabel>+107:+1001</imagelabel>
  </seclabel>
</domain>

Comment 2 Maxime Coquelin 2019-08-06 07:40:08 UTC
Hi Li,

Are you sure it should work like this?
If I understand correctly, in this scenario, the VF is bound to OVS and not to the guest.
What is bound to the guest is the Virtio device, so when setting MAC address on eth0 in guest,
it sets the MAC address for the Virtio device, not the Intel VF.

So I think this behaviour is expected, do you confirm?

Comment 3 liting 2019-08-14 02:53:09 UTC
(In reply to Maxime Coquelin from comment #2)
> Hi Li,
> 
> Are you sure it should work like this?
> If I understand correctly, in this scenario, the VF is bound to OVS and not
> to the guest.
> What is bound to the guest is the Virtio device, so when setting MAC address
> on eth0 in guest,
> it sets the MAC address for the Virtio device, not the Intel VF.
> 
> So I think this behaviour is expected, do you confirm?

Hi Maxime,

I am testing the nic partitioning feature. NIC partitioning is basically running OVS-DPDK on top of a VF. It need to test trust on function, so I test trust on with this scenario. I am not confirm it should work well. Can you confirm it? or do you know who can confirm it? If above behaviour is expected, I will close the bug and remove this scenario testing. 


thanks,
Li Ting