Hide Forgot
Description of problem: unable to backup volumes which are attached to the instance. Version-Release number of selected component (if applicable): RHOS8 How reproducible: 100% Steps to Reproduce: 1. configure nfs backend for cinder backup service. 2. create volume. 3. attach to the instance. 4. try to take backup of volume. It will fail. <snip> [root@dhcp200-223 ~(keystone_bootcamp)]# nova volume-attach ade934be-42b7-4412-be06-42d3344ee8e1 74b10200-2b7f-424a-9c9e-d3aa8b100bfd +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdb | | id | 74b10200-2b7f-424a-9c9e-d3aa8b100bfd | | serverId | ade934be-42b7-4412-be06-42d3344ee8e1 | | volumeId | 74b10200-2b7f-424a-9c9e-d3aa8b100bfd | +----------+--------------------------------------+ [root@dhcp200-223 ~(keystone_bootcamp)]# cinder backup-create 74b10200-2b7f-424a-9c9e-d3aa8b100bfd --name test3_vol_backup --force +-----------+--------------------------------------+ | Property | Value | +-----------+--------------------------------------+ | id | 4bf00dc9-24a4-4079-a6c7-26776b9af089 | | name | test3_vol_backup | | volume_id | 74b10200-2b7f-424a-9c9e-d3aa8b100bfd | +-----------+--------------------------------------+ [root@dhcp200-223 ~(keystone_bootcamp)]# cinder backup-show 4bf00dc9-24a4-4079-a6c7-26776b9af089 +-----------------------+------------------------------------------------------------+ | Property | Value | +-----------------------+------------------------------------------------------------+ | availability_zone | None | | container | None | | created_at | 2016-04-30T19:07:25.000000 | | description | None | | fail_reason | 'NfsDriver' object has no attribute 'create_cloned_volume' | | has_dependent_backups | False | | id | 4bf00dc9-24a4-4079-a6c7-26776b9af089 | | is_incremental | False | | name | test3_vol_backup | | object_count | 0 | | size | 1 | | status | error | | volume_id | 74b10200-2b7f-424a-9c9e-d3aa8b100bfd | +-----------------------+------------------------------------------------------------+ 2016-04-30 15:07:25.826 15419 INFO cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Create backup started, backup: 4bf00dc9 -24a4-4079-a6c7-26776b9af089 volume: 74b10200-2b7f-424a-9c9e-d3aa8b100bfd. 2016-04-30 15:07:25.834 15419 DEBUG cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Checking hostname 'dhcp200-223@product ion' for backend info. _get_volume_backend /usr/lib/python2.7/site-packages/cinder/backup/manager.py:116 2016-04-30 15:07:25.835 15419 DEBUG cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Got backend 'production'. _get_volume_ backend /usr/lib/python2.7/site-packages/cinder/backup/manager.py:120 2016-04-30 15:07:25.855 15419 DEBUG cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Driver requested for volume_backend 'N one'. _get_driver /usr/lib/python2.7/site-packages/cinder/backup/manager.py:146 2016-04-30 15:07:25.855 15419 DEBUG cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Fetching default backend. _get_driver /usr/lib/python2.7/site-packages/cinder/backup/manager.py:148 2016-04-30 15:07:25.856 15419 INFO cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Backend not found in hostname (None) so using default. 2016-04-30 15:07:25.856 15419 DEBUG cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Manager requested for volume_backend ' lvm'. _get_manager /usr/lib/python2.7/site-packages/cinder/backup/manager.py:134 2016-04-30 15:07:25.857 15419 DEBUG oslo_concurrency.processutils [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Running cmd (subprocess): moun t execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:230 2016-04-30 15:07:25.870 15419 DEBUG oslo_concurrency.processutils [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] CMD "mount" returned: 0 in 0.0 13s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:260 2016-04-30 15:07:25.871 15419 INFO os_brick.remotefs.remotefs [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Already mounted: /var/lib/cinder/b ackup_mount/86644d4ea62953499aa05c14c20d6b1e 2016-04-30 15:07:25.871 15419 DEBUG cinder.backup.drivers.nfs [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Using NFS backup repository: /var/lib/cinder/backup_mount/86644d4ea62953499aa05c14c20d6b1e __init__ /usr/lib/python2.7/site-packages/cinder/backup/drivers/nfs.py:58 2016-04-30 15:07:25.873 15419 DEBUG cinder.backup.drivers.posix [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Using backup repository: /var/lib/cinder/backup_mount/86644d4ea62953499aa05c14c20d6b1e __init__ /usr/lib/python2.7/site-packages/cinder/backup/drivers/posix.py:84 2016-04-30 15:07:25.874 15419 DEBUG cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Driver requested for volume_backend 'production'. _get_driver /usr/lib/python2.7/site-packages/cinder/backup/manager.py:146 2016-04-30 15:07:25.874 15419 DEBUG cinder.backup.manager [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Manager requested for volume_backend 'production'. _get_manager /usr/lib/python2.7/site-packages/cinder/backup/manager.py:134 2016-04-30 15:07:25.906 15419 DEBUG oslo_db.sqlalchemy.engines [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:256 2016-04-30 15:07:25.935 15419 DEBUG cinder.volume.driver [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Creating a new backup for volume volume-74b10200-2b7f-424a-9c9e-d3aa8b100bfd. _backup_volume_temp_volume /usr/lib/python2.7/site-packages/cinder/volume/driver.py:1103 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher [req-213d4460-a93d-480c-960e-a1085f10df5f 9fbade688f294569b040a223495c7825 7274fd8b20254b80a08f55f31eea3f99 - - -] Exception during message handling: 'NfsDriver' object has no attribute 'create_cloned_volume' 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher executor_callback)) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher executor_callback) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 399, in create_backup 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher self._update_backup_error(backup, context, six.text_type(err)) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__ 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/backup/manager.py", line 393, in create_backup 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher backup_service) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1094, in backup_volume 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher backup_service) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1114, in _backup_volume_temp_volume 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher context, volume) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1264, in _create_temp_cloned_volume 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher self.db.volume_destroy(context, temp_vol_ref['id']) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__ 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/cinder/volume/driver.py", line 1261, in _create_temp_cloned_volume 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher self.create_cloned_volume(temp_vol_ref, volume) 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher AttributeError: 'NfsDriver' object has no attribute 'create_cloned_volume' 2016-04-30 15:07:26.111 15419 ERROR oslo_messaging.rpc.dispatcher </snip> - If I detach the volume from instance then I was able to create backup. Actual results: unable to backup volumes which are attached to the instance Expected results: backup operation should be successful even if volume is attached to the instance. Additional info:
I suspect this functionality depends on snapshotting which is also missing in the NFSDriver. This upstream patch [1] should fix both NFSDriver snapshotting and disruptive backups issues. [1] https://review.openstack.org/#/c/147186/
In discussion with Cinder team, it is believed that this is unlikely to be targeted for 11 upstream. Please reopen this bug if it becomes a customer priority; otherwise it is not a current priority. Thanks, Elise