Bug 1459773

Summary: Order of the interfaces is not honored when using virt and SR-IOV interface
Product: Red Hat OpenStack Reporter: Itzik Brown <itbrown>
Component: openstack-novaAssignee: Eoghan Glynn <eglynn>
Status: CLOSED NOTABUG QA Contact: Joe H. Rahme <jhakimra>
Severity: high Docs Contact:
Priority: unspecified    
Version: 11.0 (Ocata)CC: berrange, dasmith, eglynn, kchamart, sbauza, sferdjao, sgordon, srevivo, vromanso
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-14 12:28:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Itzik Brown 2017-06-08 07:23:04 UTC 
Description of problem:
When launching an instance using the following:
# nova boot --flavor m1.small --image <image id> --nic net-id=<net id> --nic port-id=<port id> vm1

Where the first interface is a non SR-IOV port and the second one is SR-IOV port the order is not preserved ,i.e. The first interface of the instance is the SR-IOV port.

Version-Release number of selected component (if applicable):
openstack-nova-compute-15.0.3-3.el7ost.noarch

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 Stephen Gordon 2017-06-08 13:15:11 UTC 
(In reply to Itzik Brown from comment #0)
> Description of problem:
> When launching an instance using the following:
> # nova boot --flavor m1.small --image <image id> --nic net-id=<net id> --nic
> port-id=<port id> vm1
> 
> Where the first interface is a non SR-IOV port and the second one is SR-IOV
> port the order is not preserved ,i.e. The first interface of the instance is
> the SR-IOV port.
> 
> Version-Release number of selected component (if applicable):
> openstack-nova-compute-15.0.3-3.el7ost.noarch
> 
> How reproducible:
> 
> 
> Steps to Reproduce:
> 1.
> 2.
> 3.
> 
> Actual results:
> 
> 
> Expected results:
> 
> 
> Additional info:

I believe this is expected behavior (read: presentation and detection of device order is unpredictable even in physical systems) and a large part of why virt-device-role-tagging exists.
Comment 3 Daniel Berrangé 2017-06-14 10:27:35 UTC 
(In reply to Itzik Brown from comment #0)
> Description of problem:
> When launching an instance using the following:
> # nova boot --flavor m1.small --image <image id> --nic net-id=<net id> --nic
> port-id=<port id> vm1
> 
> Where the first interface is a non SR-IOV port and the second one is SR-IOV
> port the order is not preserved ,i.e. The first interface of the instance is
> the SR-IOV port.

When libvirt assigns PCI addresses to devices it iterates over each type of device. SRIOV devices are expressed as host-assigned devices, whereas non-SRIOV devices are expressed as NICs. Thus ordering from the nova boot command won't be honoured when assigning device addresses for these different types of NIC. Even if addresses are honoured, you can't assume the guest OS will detect them in device order. As Steve says, this is why you need to use device tagging to identify NICs.