Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1518969

Summary:	Failure to detach volume from a controller node after a backup of a cinder volume with LVM signature
Product:	Red Hat OpenStack	Reporter:	Mark Jones <marjones>
Component:	openstack-cinder	Assignee:	Eric Harney <eharney>
Status:	CLOSED NOTABUG	QA Contact:	Avi Avraham <aavraham>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	10.0 (Newton)	CC:	cschwede, marjones, scohen, srevivo, tshefi
Target Milestone:	---	Flags:	tshefi: automate_bug-
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-02-14 14:50:28 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1261083, 1883643, 1884225
Bug Blocks:

Description Mark Jones 2017-11-29 21:35:42 UTC

Description of problem:

When backing up a cinder volume where the volume has a LVM signature, the source volume can get stuck in detaching on the controller node causing the backup to fail:

+-----------------------+-----------------------------------------------------------------------------------------------------------------+
| Field                 | Value                                                                                                           |
+-----------------------+-----------------------------------------------------------------------------------------------------------------+
| availability_zone     | nova                                                                                                            |
| container             | 84/8c/848cbb6d-9dce-4cd3-9340-29575500d3ce                                                                      |
| created_at            | 2017-11-29T17:24:04.000000                                                                                      |
| data_timestamp        | 2017-11-29T17:22:27.000000                                                                                      |
| description           |                                                                |
| fail_reason           | Unexpected error while running command.                                                                         |
|                       | Command: multipath -f 36006016052503e00e633511a2ad5e711                                                         |
|                       | Exit code: 1                                                                                                    |
|                       | Stdout: u'Nov 29 17:31:40 | 36006016052503e00e633511a2ad5e711: map in use\nNov 29 17:31:40 | failed to remove   |
|                       | multipath map 36006016052503e00e6335                                                                            |
| has_dependent_backups | False                                                                                                           |
| id                    | 848cbb6d-9dce-4cd3-9340-29575500d3ce                                                                            |
| is_incremental        | False                                                                                                           |
| name                  | **********_volume                                                                                              |
| object_count          | 4                                                                                                               |
| size                  | 5                                                                                                               |
| snapshot_id           | e750aae5-089f-4d13-a053-aedb7edb8c2a                                                                            |
| status                | error                                                                                                           |
| updated_at            | 2017-11-29T17:32:20.000000                                                                                      |
| volume_id             | 7579ead8-3551-4b90-8485-848d84e43b00                                                                            |
+-----------------------+-----------------------------------------------------------------------------------------------------------------+
 
/var/log/cinder/backup.log:

2017-11-29 17:29:19.140 898476 WARNING oslo.privsep.daemon [req-39ee4da3-6358-404e-aa7e-8e6d9b1313de db1e6fb0f4d94a5f8f23695479780f45 7ddb6f70931a49e29811fbc1b85ba410 - default default] privsep log: 2017-11-29 17:28:42.291 946368 ERROR oslo.privsep.daemon Exit code: 1

2017-11-29 17:29:19.140 898476 WARNING oslo.privsep.daemon [req-39ee4da3-6358-404e-aa7e-8e6d9b1313de db1e6fb0f4d94a5f8f23695479780f45 7ddb6f70931a49e29811fbc1b85ba410 - default default] privsep log: 2017-11-29 17:28:42.291 946368 ERROR oslo.privsep.daemon Stdout: u'Nov 29 17:28:02 | 36006016052503e00e633511a2ad5e711: map in use\nNov 29 17:28:02 | failed to remove multipath map 36006016052503e00e633511a2ad5e711\n'

Version-Release number of selected component (if applicable):


How reproducible:

Repeatedly with Cinder volumes that contain LVM signatures

Steps to Reproduce:
1. Create a Cinder volume and write a LVM signature on it by creating a VG and LV
2. Perform a Cinder Backup operation for that volume


Actual results:

Backup errors out

Expected results:

Backup succeeds

Additional info:

This appears to be very similar to the issue in BZ 1499044 and documented in https://access.redhat.com/solutions/3213311. The customer is testing whether the use of LVM filter on the controller nodes inhibits this issue.

Comment 1 Mark Jones 2017-12-03 17:47:00 UTC

The customer has completed their testing where the LVM filter is defined as per https://access.redhat.com/solutions/3213311 and this has resolved this specific error during the Cinder Backup operation.