Bug 1862052

Summary: Unable to create VMs with SR-IOV VF when NIC Partitioning is used
Product: Red Hat OpenStack Reporter: Karthik Sundaravel <ksundara>
Component: openstack-tripleo-heat-templatesAssignee: Karthik Sundaravel <ksundara>
Status: CLOSED ERRATA QA Contact: Arik Chernetsky <achernet>
Severity: medium Docs Contact:
Priority: medium    
Version: 13.0 (Queens)CC: achernet, drosenfe, hakhande, kfida, mburns, slinaber, supadhya, vkhitrin
Target Milestone: z13Keywords: TechPreview, Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-8.4.1-66.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1859182 Environment:
Last Closed: 2020-10-28 18:24:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1859182, 1873109    
Bug Blocks:    

Description Karthik Sundaravel 2020-07-30 10:06:52 UTC
+++ This bug was initially created as a clone of Bug #1859182 +++

Templates:

  NovaPCIPassthrough:
    - devname: "p7p3"
      trusted: "true"
      physical_network: "sriov-1"
    - devname: "p7p4"
      trusted: "true"
      physical_network: "sriov-2"
    - devname: "p6p1"
      trusted: "true"
  NeutronPhysicalDevMappings:
    - sriov-1:p7p3
    - sriov-2:p7p4

The NIC configs uses NIC partitioning on p7p3 and p7p4, with VFs 0 and 1 each.

The guests are created using the below commands

openstack network create sriov1 --provider-network-type vlan --provider-physical-network sriov1 --provider-segment 502
openstack subnet create sriov1 --subnet-range 192.168.3.0/24 --network sriov1
openstack port create sriov1 --network sriov1 --vnic-type direct
sriov1_port=$(openstack port show sriov1 -c id -f value)
openstack server create --flavor m1.nfv --image rhel_nfv --nic port-id=$sriov1_port --key-name test sriov11

The guest sriov11 is in error state due to "No valid host found"

--- Additional comment from Karthik Sundaravel on 2020-07-21 12:10:48 UTC ---

Its found that the passthrough_whitelist generated in nova.conf has the PCI addresses of the applicable VFs, their vendor id and product ids.

However we find that with physical_network details, nova could reserve a VF and the guest creation is possible.
The below configuration works and the corresponding patch shall be made in pci_passthrough_whitelist.json 

passthrough_whitelist={"physical_network":"sriov-10","trusted":"true","address":{"slot":"02","bus":"82","domain":".*","function":"0"}}
passthrough_whitelist={"physical_network":"sriov-10","trusted":"true","address":{"slot":"02","bus":"82","domain":".*","function":"1"}}
passthrough_whitelist={"physical_network":"sriov-10","trusted":"true","address":{"slot":"02","bus":"82","domain":".*","function":"2"}}
passthrough_whitelist={"physical_network":"sriov-10","trusted":"true","address":{"slot":"02","bus":"82","domain":".*","function":"3"}}
passthrough_whitelist={"physical_network":"sriov-10","trusted":"true","address":{"slot":"02","bus":"82","domain":".*","function":"4"}}
passthrough_whitelist={"physical_network":"sriov-10","trusted":"true","address":{"slot":"02","bus":"82","domain":".*","function":"5"}}
passthrough_whitelist={"physical_network":"sriov-20","trusted":"true","address":{"slot":"06","bus":"82","domain":".*","function":"0"}}
passthrough_whitelist={"physical_network":"sriov-20","trusted":"true","address":{"slot":"06","bus":"82","domain":".*","function":"1"}}
passthrough_whitelist={"physical_network":"sriov-20","trusted":"true","address":{"slot":"06","bus":"82","domain":".*","function":"2"}}
passthrough_whitelist={"physical_network":"sriov-20","trusted":"true","address":{"slot":"06","bus":"82","domain":".*","function":"3"}}
passthrough_whitelist={"physical_network":"sriov-20","trusted":"true","address":{"slot":"06","bus":"82","domain":".*","function":"4"}}
passthrough_whitelist={"physical_network":"sriov-20","trusted":"true","address":{"slot":"06","bus":"82","domain":".*","function":"5"}}

--- Additional comment from Karthik Sundaravel on 2020-07-21 12:13:29 UTC ---

(In reply to Karthik Sundaravel from comment #1)
> Its found that the passthrough_whitelist generated in nova.conf has the PCI
> addresses of the applicable VFs, their vendor id and product ids.
> 
> However we find that with physical_network details, nova could reserve a VF
> and the guest creation is possible.
> The below configuration works and the corresponding patch shall be made in

derive_pci_passthrough_whitelist.py

> 
> passthrough_whitelist={"physical_network":"sriov-10","trusted":"true",
> "address":{"slot":"02","bus":"82","domain":".*","function":"0"}}
> passthrough_whitelist={"physical_network":"sriov-10","trusted":"true",
> "address":{"slot":"02","bus":"82","domain":".*","function":"1"}}
> passthrough_whitelist={"physical_network":"sriov-10","trusted":"true",
> "address":{"slot":"02","bus":"82","domain":".*","function":"2"}}
> passthrough_whitelist={"physical_network":"sriov-10","trusted":"true",
> "address":{"slot":"02","bus":"82","domain":".*","function":"3"}}
> passthrough_whitelist={"physical_network":"sriov-10","trusted":"true",
> "address":{"slot":"02","bus":"82","domain":".*","function":"4"}}
> passthrough_whitelist={"physical_network":"sriov-10","trusted":"true",
> "address":{"slot":"02","bus":"82","domain":".*","function":"5"}}
> passthrough_whitelist={"physical_network":"sriov-20","trusted":"true",
> "address":{"slot":"06","bus":"82","domain":".*","function":"0"}}
> passthrough_whitelist={"physical_network":"sriov-20","trusted":"true",
> "address":{"slot":"06","bus":"82","domain":".*","function":"1"}}
> passthrough_whitelist={"physical_network":"sriov-20","trusted":"true",
> "address":{"slot":"06","bus":"82","domain":".*","function":"2"}}
> passthrough_whitelist={"physical_network":"sriov-20","trusted":"true",
> "address":{"slot":"06","bus":"82","domain":".*","function":"3"}}
> passthrough_whitelist={"physical_network":"sriov-20","trusted":"true",
> "address":{"slot":"06","bus":"82","domain":".*","function":"4"}}
> passthrough_whitelist={"physical_network":"sriov-20","trusted":"true",
> "address":{"slot":"06","bus":"82","domain":".*","function":"5"}}

Comment 21 errata-xmlrpc 2020-10-28 18:24:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 13.0 director bug fix advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4388