Bug 1856468
| Summary: | [OSP16.1] Unable to set trust mode "ON" in SRIOV-VF in OVS-Hardware-Offload Setup | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Pradipta Kumar Sahoo <psahoo> |
| Component: | openstack-nova | Assignee: | OSP DFG:Compute <osp-dfg-compute> |
| Status: | CLOSED NOTABUG | QA Contact: | OSP DFG:Compute <osp-dfg-compute> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 16.1 (Train) | CC: | atheurer, dasmith, eglynn, fbaudin, hakhande, jhakimra, kchamart, sbauza, sgordon, vromanso |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-07-15 14:11:45 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Any changes to VF properties (Trust mode, VLAN etc) are not allowed once NIC is in "switchdev" mode. This limitation is from driver itself and below explained it. [root@nfv-compute-rt-offload-0 ~]# ip link set ens3f1 vf 2 trust on RTNETLINK answers: Operation not supported You can reverse back nic mode to "Legacy" and try this option, it should work. Either the CX5 is used as OVS HW offload, and there is no SR-IOV trusted mode, either it is used as a legacy SR-IOV NIC, and there is no OVS HW offload. The fact that OVS HW offload looks like SR-IOV is confusing however it is not SR-IOV ;-) So are we stating that with using OVS-HW-offload, there is no way to have the VF receive packets which have many different destination MACs? The issue is not setting "trust on". The issue is setting the right OVS flows to have the same behavior as having trust-on in legacy mode. Trust-on just happens to be how to accomplish this functionality with legacy SRIOV. We should still be able to accomplish the same functionality, but with a different implementation with OVS-hw-offload. Andrew, Trust mode allows mac address change from instance. Does testcase here to change VF's mac? Spoof check drops packets where non VF's mac is source mac. I think test case is, From VF, you are sending multiple different source mac packets and they are dropped. Is that the case? If yes, then disable neutron port security for port. so packets wont be dropped. if they are still dropped, then neutron port security are not applying on VFs and its a bug. Trust mode allowed a packet with any destination MAC to be sent to the VF. This is what we need. We also need to disable port security to allow sending packets from the VM with different src MACs. Wha we are doing is emulating a L2 switch in the VM, and that requires both receiving and sending packets with many different src/dest MACs unmodified. We have done this successfully with regular SRIOV. In order for OVS-HW-offload to do this, we would need the proper flows programmed for this provider network, and not any flows which only allowed packets to be received where the dest MAC matched the VF's MAC. |
Description of problem: The "TRUST" option doesn't reflect in SRIOV-VF after instance provision with bind profile. Version-Release number of selected component (if applicable): Red Hat OpenStack Platform release 16.1.0 RC (Train) How reproducible: 100% reproducible in NFV perf lab Steps to Reproduce: 1. Deployed overcloud environment with below nova pci-pt features. ComputeSriovRTParameters: ... NovaPCIPassthrough: - devname: "ens3f0" trusted: "true" physical_network: "offload1" - devname: "ens3f1" trusted: "true" physical_network: "offload2" 2. Nova cnfiguration updated with trust mode in post deployment validation. # grep ^[^#] /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf | grep passthrough_whitelist passthrough_whitelist={"devname":"ens3f0","physical_network":"offload1","trusted":"true"} passthrough_whitelist={"devname":"ens3f1","physical_network":"offload2","trusted":"true"} 3. Existing neutron offload port with set with additional binding profile with trust mode. (overcloud) [stack@ospd16-1 ~]$ nova interface-list TestPMD +------------+--------------------------------------+--------------------------------------+-----------------+-------------------+-----+ | Port State | Port ID | Net ID | IP addresses | MAC Addr | Tag | +------------+--------------------------------------+--------------------------------------+-----------------+-------------------+-----+ | ACTIVE | 601d0399-b98a-4751-932f-673021e44c7d | e91bbe4f-43d6-4330-b47d-1420353f75d4 | 192.168.176.50 | fa:16:3e:22:65:90 | - | | ACTIVE | 62bfdc26-d91f-4d49-b96d-6e0a843f2234 | 679933fc-4810-402e-a0c1-dc21243fa991 | 192.168.178.123 | fa:16:3e:72:b0:ea | - | | ACTIVE | 7fb2741c-631a-4e64-820b-77fd976ab268 | 3cb87592-85dd-41ed-a4b7-4101f99bff43 | 192.168.177.194 | fa:16:3e:ff:3d:9d | - | +------------+--------------------------------------+--------------------------------------+-----------------+-------------------+-----+ (overcloud) [stack@ospd16-1 ~]$ openstack port show --fit-width 62bfdc26-d91f-4d49-b96d-6e0a843f2234 +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | nfv-compute-rt-offload-0.localdomain | | binding_profile | capabilities='['switchdev']', pci_slot='0000:5e:01.0', pci_vendor_info='15b3:101a', physical_network='offload2', trusted='true' | | binding_vif_details | bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='True', port_filter='True' | | binding_vif_type | ovs | | binding_vnic_type | direct | | created_at | 2020-07-12T16:19:38Z | | data_plane_status | None | | description | | | device_id | 981a9856-7acb-4cbd-bab6-a9700f756a4d | | device_owner | compute:nfvprovider | | dns_assignment | None | | dns_domain | None | | dns_name | None | | extra_dhcp_opts | | | fixed_ips | ip_address='192.168.178.123', subnet_id='2726ad12-5c23-488f-85a7-46fa90a99cc1' | | id | 62bfdc26-d91f-4d49-b96d-6e0a843f2234 | | location | cloud='', project.domain_id=, project.domain_name='Default', project.id='0532f685d6084767baea7960e80f5a00', project.name='admin', region_name='regionOne', zone= | | mac_address | fa:16:3e:72:b0:ea | | name | provider2-port1 | | network_id | 679933fc-4810-402e-a0c1-dc21243fa991 | | port_security_enabled | False | | project_id | 0532f685d6084767baea7960e80f5a00 | | propagate_uplink_status | None | | qos_policy_id | None | | resource_request | None | | revision_number | 12 | | security_group_ids | | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2020-07-12T19:13:57Z | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [root@nfv-compute-rt-offload-0 ~]# ip link show ens3f1 28: ens3f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000 link/ether ec:0d:9a:44:2c:f5 brd ff:ff:ff:ff:ff:ff vf 0 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state disable, trust off, query_rss off vf 1 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state disable, trust off, query_rss off vf 2 link/ether fa:16:3e:72:b0:ea brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state disable, trust off, query_rss off vf 3 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state disable, trust off, query_rss off 4. The manual modification also doesn't allow the operation. [root@nfv-compute-rt-offload-0 ~]# ip link set ens3f0 vf 2 trust on RTNETLINK answers: Operation not supported [root@nfv-compute-rt-offload-0 ~]# ip link set ens3f1 vf 2 trust on RTNETLINK answers: Operation not supported Actual results: Trust mode failed to setup Expected results: Trust mode should work as it configured in nova passthrough_whitelist section. BR, Pradipta