Bug 1489087

Summary:	Backport patch to RHOSP 10 for EMC VNX backups
Product:	Red Hat OpenStack	Reporter:	David Hill <dhill>
Component:	openstack-cinder	Assignee:	Eric Harney <eharney>
Status:	CLOSED ERRATA	QA Contact:	Avi Avraham <aavraham>
Severity:	high	Docs Contact:
Priority:	high
Version:	10.0 (Newton)	CC:	cswanson, dcadzow, dhill, eharney, gkeegan, marjones, mlinden, mlopes, pgrist, slinaber, srevivo, tshefi
Target Milestone:	z7	Keywords:	FeatureBackport, OtherQA, Triaged, ZStream
Target Release:	10.0 (Newton)	Flags:	tshefi: automate_bug-
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-cinder-9.1.4-14.el7ost	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-02-27 16:39:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description David Hill 2017-09-06 16:24:35 UTC

Description of problem:
Backport patch [1] to RHOSP 10 for EMC VNX backups generating the following traces:
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 404, in create_backup
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     self._update_backup_error(backup, six.text_type(err))
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 398, in create_backup
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     self._run_backup(context, backup, volume)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 434, in _run_backup
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     properties, is_snapshot)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 910, in _attach_device
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     context, backup_device, properties)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1053, in _attach_snapshot
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     return self._connect_device(conn)
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1059, in _connect_device
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server     protocol = conn['driver_volume_type']
2017-09-06 08:51:26.039 37336 ERROR oslo_messaging.rpc.server TypeError: 'NoneType' object has no attribute '__getitem__'


[1] https://bugs.launchpad.net/cinder/+bug/1693956

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Try backing up a cinder volume that is attached to an instance
2.
3.

Actual results:
Fails

Expected results:
Succeeds

Additional info:

Comment 1 David Hill 2017-09-24 14:29:21 UTC

We'll later on hit this message when trying to delete the volume:

2017-09-17 11:47:25.732 96354 ERROR oslo_messaging.rpc.server SP A: The LUN cannot be destroyed because it still has snapshot mount points. Destroy the snapshot mount points then retry the LUN destroy command. (0x716d801a)

Comment 2 David Hill 2017-09-24 14:29:51 UTC

and this one:

2017-09-21 00:10:00.801 672172 ERROR cinder.volume.driver VNXLunInStorageGroupError: failed to remove lun tmp-smp-0e06ab62-e27b-4079-8608-6207f034e640  detail:
2017-09-21 00:10:00.801 672172 ERROR cinder.volume.driver Cannot unbind LUN because it's contained in a Storage Group

Comment 12 errata-xmlrpc 2018-02-27 16:39:47 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0360