Looks like we are not able to backup NFS storage when adding --snapshot-id Steps: Create a volume VOL Create a snapshot from the volume VOL - [name: SNAP] Create a backup VOL --snapshot-id SNAP ---> Fails. Is it a known issue? ( i see that the same test works on CEPH storage) The error from NFS is "cinder.exception.InvalidSnapshot: Snapshot status must be "available" to clone. But is backing-up" info : test volume_backup_with_snapshot --fails NFS (overcloud) [stack@undercloud-0 tempest-dir]$ cinder show 5bea6aa6-cc2b-4a62-9074-9ce1375218c0 +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attached_servers | [] | | attachment_ids | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2020-10-29T07:14:42.000000 | | description | None | | encrypted | False | | id | 5bea6aa6-cc2b-4a62-9074-9ce1375218c0 | | metadata | | | migration_status | None | | multiattach | False | | name | test | | os-vol-host-attr:host | hostgroup@tripleo_nfs#tripleo_nfs | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5a491aeebb514f8f99ede8e9003bc1a9 | | replication_status | None | | size | 1 | | snapshot_id | None | | source_volid | None | | status | available | | updated_at | 2020-10-29T07:14:43.000000 | | user_id | 76a40a7d587d4d058b59fd9ae3128132 | | volume_type | tripleo | +--------------------------------+--------------------------------------+ (overcloud) [stack@undercloud-0 tempest-dir]$ cinder snapshot-show 0381cde5-e810-437a-a292-a8059ff03508 +--------------------------------------------+--------------------------------------+ | Property | Value | +--------------------------------------------+--------------------------------------+ | created_at | 2020-10-29T07:15:17.000000 | | description | None | | id | 0381cde5-e810-437a-a292-a8059ff03508 | | metadata | {} | | name | test | | os-extended-snapshot-attributes:progress | 100% | | os-extended-snapshot-attributes:project_id | 5a491aeebb514f8f99ede8e9003bc1a9 | | size | 1 | | status | available | | updated_at | 2020-10-29T07:18:25.000000 | | volume_id | 5bea6aa6-cc2b-4a62-9074-9ce1375218c0 | +--------------------------------------------+--------------------------------------+ # cinder backup-create 5bea6aa6-cc2b-4a62-9074-9ce1375218c0 --snapshot-id 0381cde5-e810-437a-a292-a8059ff03508 -- >Fails . (overcloud) [stack@undercloud-0 tempest-dir]$ cinder backup-list +--------------------------------------+--------------------------------------+--------+------+------+--------------+-----------+ | ID | Volume ID | Status | Name | Size | Object Count | Container | +--------------------------------------+--------------------------------------+--------+------+------+--------------+-----------+ | 4235d937-611a-40cb-b2dc-2d4f33aee9fb | 5bea6aa6-cc2b-4a62-9074-9ce1375218c0 | error | - | 1 | 0 | - | +--------------------------------------+--------------------------------------+--------+------+------+--------------+-----------+ ==== controller-1 controllers ==== /var/log/containers/cinder/cinder-backup.log:16:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server [req-c20051cd-29ad-4221-b8f1-c949512c64e1 76a40a7d587d4d058b59fd9ae3128132 5a491aeebb514f8f99ede8e9003bc1a9 - default default] Exception during message handling: cinder.exception_Remote.InvalidSnapshot_Remote: Snapshot status must be "available" to clone. But is: backing-up /var/log/containers/cinder/cinder-backup.log:64:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server Traceback (most recent call last): /var/log/containers/cinder/cinder-backup.log:65:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming /var/log/containers/cinder/cinder-backup.log:66:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) /var/log/containers/cinder/cinder-backup.log:67:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch /var/log/containers/cinder/cinder-backup.log:68:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) /var/log/containers/cinder/cinder-backup.log:69:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch /var/log/containers/cinder/cinder-backup.log:70:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) /var/log/containers/cinder/cinder-backup.log:71:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 417, in create_backup /var/log/containers/cinder/cinder-backup.log:72:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server self._update_backup_error(backup, six.text_type(err)) /var/log/containers/cinder/cinder-backup.log:73:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ /var/log/containers/cinder/cinder-backup.log:74:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server self.force_reraise() /var/log/containers/cinder/cinder-backup.log:75:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise /var/log/containers/cinder/cinder-backup.log:76:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) /var/log/containers/cinder/cinder-backup.log:77:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise /var/log/containers/cinder/cinder-backup.log:78:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server raise value /var/log/containers/cinder/cinder-backup.log:79:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 406, in create_backup /var/log/containers/cinder/cinder-backup.log:80:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server updates = self._run_backup(context, backup, volume) /var/log/containers/cinder/cinder-backup.log:81:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 473, in _run_backup /var/log/containers/cinder/cinder-backup.log:82:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server volume) /var/log/containers/cinder/cinder-backup.log:83:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/rpcapi.py", line 347, in get_backup_device /var/log/containers/cinder/cinder-backup.log:84:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server want_objects=True) /var/log/containers/cinder/cinder-backup.log:85:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 181, in call /var/log/containers/cinder/cinder-backup.log:86:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server transport_options=self.transport_options) /var/log/containers/cinder/cinder-backup.log:87:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/transport.py", line 129, in _send /var/log/containers/cinder/cinder-backup.log:88:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server transport_options=transport_options) /var/log/containers/cinder/cinder-backup.log:89:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 646, in send /var/log/containers/cinder/cinder-backup.log:90:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server transport_options=transport_options) /var/log/containers/cinder/cinder-backup.log:91:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send /var/log/containers/cinder/cinder-backup.log:92:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server raise result /var/log/containers/cinder/cinder-backup.log:93:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server cinder.exception_Remote.InvalidSnapshot_Remote: Snapshot status must be "available" to clone. But is: backing-up /var/log/containers/cinder/cinder-backup.log:94:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server Traceback (most recent call last): /var/log/containers/cinder/cinder-backup.log:95:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:96:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming /var/log/containers/cinder/cinder-backup.log:97:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) /var/log/containers/cinder/cinder-backup.log:98:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:99:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch /var/log/containers/cinder/cinder-backup.log:100:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) /var/log/containers/cinder/cinder-backup.log:101:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:102:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch /var/log/containers/cinder/cinder-backup.log:103:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) /var/log/containers/cinder/cinder-backup.log:104:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:105:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/manager.py", line 4447, in get_backup_device /var/log/containers/cinder/cinder-backup.log:106:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server self.driver.get_backup_device(ctxt, backup)) /var/log/containers/cinder/cinder-backup.log:107:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:108:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/driver.py", line 1192, in get_backup_device /var/log/containers/cinder/cinder-backup.log:109:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server context, backup) /var/log/containers/cinder/cinder-backup.log:110:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:111:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/driver.py", line 1220, in _get_backup_volume_temp_volume /var/log/containers/cinder/cinder-backup.log:112:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server context, volume, snapshot) /var/log/containers/cinder/cinder-backup.log:113:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:114:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/driver.py", line 1357, in _create_temp_volume_from_snapshot /var/log/containers/cinder/cinder-backup.log:115:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server temp_vol_ref.destroy() /var/log/containers/cinder/cinder-backup.log:116:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:117:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ /var/log/containers/cinder/cinder-backup.log:118:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server self.force_reraise() /var/log/containers/cinder/cinder-backup.log:119:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:120:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise /var/log/containers/cinder/cinder-backup.log:121:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) /var/log/containers/cinder/cinder-backup.log:122:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:123:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise /var/log/containers/cinder/cinder-backup.log:124:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server raise value /var/log/containers/cinder/cinder-backup.log:125:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:126:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/driver.py", line 1352, in _create_temp_volume_from_snapshot /var/log/containers/cinder/cinder-backup.log:127:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server snapshot) /var/log/containers/cinder/cinder-backup.log:128:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:129:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "<decorator-gen-249>", line 2, in create_volume_from_snapshot /var/log/containers/cinder/cinder-backup.log:130:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:131:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/coordination.py", line 151, in _synchronized /var/log/containers/cinder/cinder-backup.log:132:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server return f(*a, **k) /var/log/containers/cinder/cinder-backup.log:133:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:134:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/remotefs.py", line 1699, in create_volume_from_snapshot /var/log/containers/cinder/cinder-backup.log:135:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server return self._create_volume_from_snapshot(volume, snapshot) /var/log/containers/cinder/cinder-backup.log:136:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:137:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/volume/drivers/remotefs.py", line 1265, in _create_volume_from_snapshot /var/log/containers/cinder/cinder-backup.log:138:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server raise exception.InvalidSnapshot(msg) /var/log/containers/cinder/cinder-backup.log:139:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:140:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server cinder.exception.InvalidSnapshot: Snapshot status must be "available" to clone. But is: backing-up /var/log/containers/cinder/cinder-backup.log:141:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server /var/log/containers/cinder/cinder-backup.log:142:2020-10-29 07:22:19.043 24 ERROR oslo_messaging.rpc.server Warning: Permanently added '192.168.24.19' (ECDSA) to the list of known hosts. This system is not registered to Red Hat Insights. See https://cloud.redhat.com/ To register this system, run: insights-client --register
Verified on: openstack-cinder-15.3.1-7.el8ost.noarch Create a volume on NFS backend: (overcloud) [stack@undercloud-0 ~]$ cinder create 3 --name volA +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2021-04-15T17:13:30.000000 | | description | None | | encrypted | False | | id | a99f7934-cae5-45e4-bc40-50a0d37b0a95 | | metadata | {} | | migration_status | None | | multiattach | False | | name | volA | | os-vol-host-attr:host | hostgroup@tripleo_nfs#tripleo_nfs | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 8600897b0a8b4f9d8c333217b0c8cb21 | | replication_status | None | | size | 3 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | 2021-04-15T17:13:30.000000 | | user_id | 9bce32ace9eb467880ce6042638d9b1b | | volume_type | tripleo | +--------------------------------+--------------------------------------+ Attach vol to instance, wrote data, detach volume. Create snapshot of volume: (overcloud) [stack@undercloud-0 ~]$ cinder snapshot-create a99f7934-cae5-45e4-bc40-50a0d37b0a95 --name span1 +-------------+--------------------------------------+ | Property | Value | +-------------+--------------------------------------+ | created_at | 2021-04-15T18:32:36.941419 | | description | None | | id | 75bc4935-be03-4985-a33b-8582fb45704f | | metadata | {} | | name | span1 | | size | 3 | | status | creating | | updated_at | None | | volume_id | a99f7934-cae5-45e4-bc40-50a0d37b0a95 | +-------------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder snapshot-list +--------------------------------------+--------------------------------------+-----------+-------+------+ | ID | Volume ID | Status | Name | Size | +--------------------------------------+--------------------------------------+-----------+-------+------+ | 75bc4935-be03-4985-a33b-8582fb45704f | a99f7934-cae5-45e4-bc40-50a0d37b0a95 | available | span1 | 3 | +--------------------------------------+--------------------------------------+-----------+-------+------+ Backup the snapshot: (overcloud) [stack@undercloud-0 ~]$ cinder backup-create a99f7934-cae5-45e4-bc40-50a0d37b0a95 --snapshot-id 75bc4935-be03-4985-a33b-8582fb45704f --name SnapshotBackup +-----------+--------------------------------------+ | Property | Value | +-----------+--------------------------------------+ | id | 3cf9fa4d-61fa-4f75-8c1c-b54a6321cc35 | | name | SnapshotBackup | | volume_id | a99f7934-cae5-45e4-bc40-50a0d37b0a95 | +-----------+--------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ cinder backup-list +--------------------------------------+--------------------------------------+----------+----------------+------+--------------+-----------+ | ID | Volume ID | Status | Name | Size | Object Count | Container | +--------------------------------------+--------------------------------------+----------+----------------+------+--------------+-----------+ | 3cf9fa4d-61fa-4f75-8c1c-b54a6321cc35 | a99f7934-cae5-45e4-bc40-50a0d37b0a95 | creating | SnapshotBackup | 3 | 0 | - | +--------------------------------------+--------------------------------------+----------+----------------+------+--------------+-----------+ Backup is successfully created this time: (overcloud) [stack@undercloud-0 ~]$ cinder backup-list +--------------------------------------+--------------------------------------+-----------+----------------+------+--------------+---------------+ | ID | Volume ID | Status | Name | Size | Object Count | Container | +--------------------------------------+--------------------------------------+-----------+----------------+------+--------------+---------------+ | 3cf9fa4d-61fa-4f75-8c1c-b54a6321cc35 | a99f7934-cae5-45e4-bc40-50a0d37b0a95 | available | SnapshotBackup | 3 | 62 | volumebackups | +--------------------------------------+--------------------------------------+-----------+----------------+------+--------------+---------------+ Good to verify.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.6 bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2097