Bug 1274054

Summary: When using a FCoE adapter instead of a FC adapter, volumes will fail to attach to the VM
Product: Red Hat OpenStack Reporter: David Hill <dhill>
Component: openstack-novaAssignee: Lee Yarwood <lyarwood>
Status: CLOSED ERRATA QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: high    
Version: 6.0 (Juno)CC: berrange, byount, dasmith, dhill, eglynn, kchamart, lyarwood, ndipanov, pbrady, rbiba, sbauza, sferdjao, sgordon, vromanso, yeylon
Target Milestone: z3Keywords: OtherQA, Triaged, ZStream
Target Release: 7.0 (Kilo)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-nova-2015.1.2-7.el7ost Doc Type: Bug Fix
Doc Text:
FCoE devices have different sysfs paths to standard FC devices. As a consequence, Nova previously failed when attempting to attach an FCoE based volume to an instance as it assumed these paths were the same. This update corrects the _get_pci_num method to parse the required PCI information from both FC and FCoE sysfs device paths. As a result, Nova now succeeds in attaching FCoE based volumes to instances.
Story Points: ---
Clone Of:
: 1284033 (view as bug list) Environment:
Last Closed: 2015-12-21 17:07:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1284033    

Description David Hill 2015-10-21 19:10:21 UTC
Description of problem:
When using a FCoE adapter instead of a FC adapter, volumes will fail to attach to the VM .  The issue has been pinpointed to the way the libvirt drivers are handling the path of the created device in order to extract the PCI location of the device .   In order to support FCoE, the following patch needs to be applied:

[root@picompute02 libvirt]# diff -u volume.py.old volume.py
--- volume.py.old	2015-10-21 14:47:28.744189152 -0400
+++ volume.py	2015-10-21 14:07:18.317114439 -0400
@@ -997,6 +997,7 @@
         # NOTE(walter-boring)
         # device path is in format of
         # /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2
+        # /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2/host3/fc_host/host3
         # sometimes an extra entry exists before the host2 value
         # we always want the value prior to the host2 value
         pci_num = None
@@ -1010,7 +1011,7 @@
                     index = index + 1
 
                 if index > 0:
-                    pci_num = device_path[index - 1]
+                    pci_num = device_path[index - 4]
 
         return pci_num


This is still an ugly patch because it breaks support for regular FC adapters.  For what it worths, I think this patch might be better:

--- a/nova/virt/libvirt/volume.py
+++ b/nova/virt/libvirt/volume.py
@@ -990,12 +990,12 @@ class LibvirtFibreChannelVolumeDriver(LibvirtBaseVolumeDriver):
                 index = 0
                 device_path = hba['device_path'].split('/')
                 for value in device_path:
-                    if value.startswith('host'):
+                    if value.startswith('pci'):
                         break
                     index = index + 1
 
                 if index > 0:
-                    pci_num = device_path[index - 1]
+                    pci_num = device_path[index + 3]
 
         return pci_num


Version-Release number of selected component (if applicable):
Latest

How reproducible:
Always

Steps to Reproduce:
1. Configure 3PAR storage
2. Add computes with FCoE adapters
3. Try to attach a volume

Actual results:
Will fail to attach

Expected results:
Should attach


Additional info:

This is due to the device path being different when added through FCoE instead of regular FC devices

FC:
 /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2
FCoE:
 /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2/host3/fc_host/host3

Comment 4 Lee Yarwood 2015-10-22 13:47:25 UTC
I've not been able to formally test this but the following should allow for both FC and FCoE devices :

# git diff nova/virt/libvirt/volume.py
diff --git a/nova/virt/libvirt/volume.py b/nova/virt/libvirt/volume.py
index ef43696..6116940 100644
--- a/nova/virt/libvirt/volume.py
+++ b/nova/virt/libvirt/volume.py
@@ -995,24 +995,18 @@ class LibvirtFibreChannelVolumeDriver(LibvirtBaseVolumeDriver):
 
     def _get_pci_num(self, hba):
         # NOTE(walter-boring)
-        # device path is in format of
+        # device path is in format of (FC and FCoE) :
         # /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2
+        # /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2/host3/fc_host/host3 #noqa
         # sometimes an extra entry exists before the host2 value
         # we always want the value prior to the host2 value
-        pci_num = None
         if hba is not None:
             if "device_path" in hba:
-                index = 0
                 device_path = hba['device_path'].split('/')
-                for value in device_path:
-                    if value.startswith('host'):
-                        break
-                    index = index + 1
-
-                if index > 0:
-                    pci_num = device_path[index - 1]
-
-        return pci_num
+                for index, value in enumerate(device_path):
+                    if value.startswith('net') or value.startswith('host'):
+                        return device_path[index - 1]
+        return None
 
     def get_config(self, connection_info, disk_info):
         """Returns xml for libvirt."""

This appears to be the case in os-brick so I'll also submit this upstream for further review.

Comment 10 Lee Yarwood 2015-10-30 16:48:22 UTC
*** Bug 1274056 has been marked as a duplicate of this bug. ***

Comment 11 Lee Yarwood 2015-10-30 16:48:24 UTC
*** Bug 1274055 has been marked as a duplicate of this bug. ***

Comment 21 errata-xmlrpc 2015-12-21 17:07:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:2673