Bug 1274054 - When using a FCoE adapter instead of a FC adapter, volumes will fail to attach to the VM
When using a FCoE adapter instead of a FC adapter, volumes will fail to attac...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova (Show other bugs)
6.0 (Juno)
Unspecified Unspecified
high Severity high
: z3
: 7.0 (Kilo)
Assigned To: Lee Yarwood
nlevinki
: OtherQA, Triaged, ZStream
: 1274055 1274056 (view as bug list)
Depends On:
Blocks: 1284033
  Show dependency treegraph
 
Reported: 2015-10-21 15:10 EDT by David Hill
Modified: 2015-12-21 12:07 EST (History)
17 users (show)

See Also:
Fixed In Version: openstack-nova-2015.1.2-7.el7ost
Doc Type: Bug Fix
Doc Text:
FCoE devices have different sysfs paths to standard FC devices. As a consequence, Nova previously failed when attempting to attach an FCoE based volume to an instance as it assumed these paths were the same. This update corrects the _get_pci_num method to parse the required PCI information from both FC and FCoE sysfs device paths. As a result, Nova now succeeds in attaching FCoE based volumes to instances.
Story Points: ---
Clone Of:
: 1284033 (view as bug list)
Environment:
Last Closed: 2015-12-21 12:07:41 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1997593 None None None Never
OpenStack gerrit 238838 None None None Never

  None (edit)
Description David Hill 2015-10-21 15:10:21 EDT
Description of problem:
When using a FCoE adapter instead of a FC adapter, volumes will fail to attach to the VM .  The issue has been pinpointed to the way the libvirt drivers are handling the path of the created device in order to extract the PCI location of the device .   In order to support FCoE, the following patch needs to be applied:

[root@picompute02 libvirt]# diff -u volume.py.old volume.py
--- volume.py.old	2015-10-21 14:47:28.744189152 -0400
+++ volume.py	2015-10-21 14:07:18.317114439 -0400
@@ -997,6 +997,7 @@
         # NOTE(walter-boring)
         # device path is in format of
         # /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2
+        # /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2/host3/fc_host/host3
         # sometimes an extra entry exists before the host2 value
         # we always want the value prior to the host2 value
         pci_num = None
@@ -1010,7 +1011,7 @@
                     index = index + 1
 
                 if index > 0:
-                    pci_num = device_path[index - 1]
+                    pci_num = device_path[index - 4]
 
         return pci_num


This is still an ugly patch because it breaks support for regular FC adapters.  For what it worths, I think this patch might be better:

--- a/nova/virt/libvirt/volume.py
+++ b/nova/virt/libvirt/volume.py
@@ -990,12 +990,12 @@ class LibvirtFibreChannelVolumeDriver(LibvirtBaseVolumeDriver):
                 index = 0
                 device_path = hba['device_path'].split('/')
                 for value in device_path:
-                    if value.startswith('host'):
+                    if value.startswith('pci'):
                         break
                     index = index + 1
 
                 if index > 0:
-                    pci_num = device_path[index - 1]
+                    pci_num = device_path[index + 3]
 
         return pci_num


Version-Release number of selected component (if applicable):
Latest

How reproducible:
Always

Steps to Reproduce:
1. Configure 3PAR storage
2. Add computes with FCoE adapters
3. Try to attach a volume

Actual results:
Will fail to attach

Expected results:
Should attach


Additional info:

This is due to the device path being different when added through FCoE instead of regular FC devices

FC:
 /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2
FCoE:
 /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2/host3/fc_host/host3
Comment 4 Lee Yarwood 2015-10-22 09:47:25 EDT
I've not been able to formally test this but the following should allow for both FC and FCoE devices :

# git diff nova/virt/libvirt/volume.py
diff --git a/nova/virt/libvirt/volume.py b/nova/virt/libvirt/volume.py
index ef43696..6116940 100644
--- a/nova/virt/libvirt/volume.py
+++ b/nova/virt/libvirt/volume.py
@@ -995,24 +995,18 @@ class LibvirtFibreChannelVolumeDriver(LibvirtBaseVolumeDriver):
 
     def _get_pci_num(self, hba):
         # NOTE(walter-boring)
-        # device path is in format of
+        # device path is in format of (FC and FCoE) :
         # /sys/devices/pci0000:00/0000:00:03.0/0000:05:00.3/host2/fc_host/host2
+        # /sys/devices/pci0000:20/0000:20:03.0/0000:21:00.2/net/ens2f2/ctlr_2/host3/fc_host/host3 #noqa
         # sometimes an extra entry exists before the host2 value
         # we always want the value prior to the host2 value
-        pci_num = None
         if hba is not None:
             if "device_path" in hba:
-                index = 0
                 device_path = hba['device_path'].split('/')
-                for value in device_path:
-                    if value.startswith('host'):
-                        break
-                    index = index + 1
-
-                if index > 0:
-                    pci_num = device_path[index - 1]
-
-        return pci_num
+                for index, value in enumerate(device_path):
+                    if value.startswith('net') or value.startswith('host'):
+                        return device_path[index - 1]
+        return None
 
     def get_config(self, connection_info, disk_info):
         """Returns xml for libvirt."""

This appears to be the case in os-brick so I'll also submit this upstream for further review.
Comment 10 Lee Yarwood 2015-10-30 12:48:22 EDT
*** Bug 1274056 has been marked as a duplicate of this bug. ***
Comment 11 Lee Yarwood 2015-10-30 12:48:24 EDT
*** Bug 1274055 has been marked as a duplicate of this bug. ***
Comment 21 errata-xmlrpc 2015-12-21 12:07:41 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:2673

Note You need to log in before you can comment on or make changes to this bug.