Bug 1391680

Summary: Couldn't boot VM with PF as interface in HP environment
Product: Red Hat OpenStack Reporter: Eyal Dannon <edannon>
Component: openstack-novaAssignee: Eoghan Glynn <eglynn>
Status: CLOSED DUPLICATE QA Contact: Prasanth Anbalagan <panbalag>
Severity: high Docs Contact:
Priority: high    
Version: 10.0 (Newton)CC: atelang, atragler, berrange, dasmith, eglynn, fbaudin, jshortt, kchamart, mbabushk, oblaut, panbalag, sbauza, sferdjao, sgordon, srevivo, vromanso, yrachman
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-09 00:35:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Eyal Dannon 2016-11-03 18:29:20 UTC 
Description of problem:

Part of OSPd10 + SR-IOV verification I've tried to boot a VM with PF as interface.
I have done it on another setup(see bug 1390576), the difference is that now I'm working on HP Gen9 environment.

I configured the following as pci_whitelist:
pci_passthrough_whitelist =[{"vendor_id":"8086", "product_id":"10fb", "physical_network":"tenant"}, {"vendor_id":"8086", "product_id":"10ed", "physical_network":"tenant"} ]

The device I'm using:
05:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
05:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
05:10.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.3 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.5 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.7 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)

When I'm trying to boot an instance with direct port, the instance successfully up and active.
While using "direct-physical" port, the instance enters to ERROR state, with the following log:

| binding:vnic_type     | direct-physical      
nove-scheduler: http://pastebin.test.redhat.com/426820
nova-compute: http://pastebin.test.redhat.com/426821


Version-Release number of selected component (if applicable):


openstack-nova-novncproxy-14.0.1-5.el7ost.noarch
openstack-nova-common-14.0.1-5.el7ost.noarch
openstack-nova-scheduler-14.0.1-5.el7ost.noarch
python-nova-14.0.1-5.el7ost.noarch
openstack-nova-compute-14.0.1-5.el7ost.noarch
openstack-nova-console-14.0.1-5.el7ost.noarch
openstack-nova-cert-14.0.1-5.el7ost.noarch
python-novaclient-6.0.0-1.el7ost.noarch
openstack-nova-api-14.0.1-5.el7ost.noarch
openstack-nova-conductor-14.0.1-5.el7ost.noarch
puppet-nova-9.4.0-1.el7ost.noarch



How reproducible:
Always

Steps to Reproduce:
1.Setup OSPd10 + SR-IOV environment
2.Set the "pci_passthrough_whitelist" as mentioned above
3.Boot an instance with direct-physical port as interface

Actual results:
VM boots to ERROR state

Expected results:
The VM should boot to ACTIVE, RUNNING state


Additional info:
Comment 1 Yariv 2016-11-06 16:00:26 UTC 
We Contacted HP support

They forwarded us to latest Intel Driver
https://downloadcenter.intel.com/download/14687/Network-Adapter-Driver-for-PCI-E-Intel-10-Gigabit-Ethernet-Network-Connections-under-Linux-?product=41282

download ixgbe-4.4.6

Looking at RHEL 7.3 
driver: ixgbe
version: 4.4.0-k-rh7.3
firmware-version: 0x80000897
expansion-rom-version: 
bus-info: 0000:08:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

There are also HP drivers for RHEL7.2
https://h20566.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=5283363&swLangOid=8&swEnvOid=4176

HPE Intel ixgbevf Drivers for Red Hat Enterprise Linux 7 x86_64 (American, International)

What is the recommended guidelines to work with RHEL 7.3 Kernel  3.10.0-513.el7.x86_64?
Comment 3 Stephen Gordon 2016-11-08 13:19:12 UTC 
(In reply to Yariv from comment #1)
> We Contacted HP support
> 
> They forwarded us to latest Intel Driver
> https://downloadcenter.intel.com/download/14687/Network-Adapter-Driver-for-
> PCI-E-Intel-10-Gigabit-Ethernet-Network-Connections-under-Linux-
> ?product=41282
> 
> download ixgbe-4.4.6
> 
> Looking at RHEL 7.3 
> driver: ixgbe
> version: 4.4.0-k-rh7.3
> firmware-version: 0x80000897
> expansion-rom-version: 
> bus-info: 0000:08:00.0
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
> 
> There are also HP drivers for RHEL7.2
> https://h20566.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.
> oid=5283363&swLangOid=8&swEnvOid=4176
> 
> HPE Intel ixgbevf Drivers for Red Hat Enterprise Linux 7 x86_64 (American,
> International)
> 
> What is the recommended guidelines to work with RHEL 7.3 Kernel 
> 3.10.0-513.el7.x86_64?

Hi Yariv,

Can you elaborate on how we came to the conclusion that this is a driver issue?"

"PortBindingFailed: Binding failed for port a9d9c24a-8e05-4b30-8eab-71601bed1a62, please check neutron logs for more information."

What was in the Neutron logs? 

Thanks,

Steve
Comment 7 Stephen Gordon 2016-11-09 00:35:30 UTC 

*** This bug has been marked as a duplicate of bug 1390576 ***