Bug 1598513
| Summary: | Unable to retype from a ceph backend to a vmax backend or vice-versa | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | David Hill <dhill> |
| Component: | openstack-cinder | Assignee: | Cinder Bugs List <cinder-bugs> |
| Status: | CLOSED DUPLICATE | QA Contact: | Avi Avraham <aavraham> |
| Severity: | high | Docs Contact: | Kim Nylander <knylande> |
| Priority: | high | ||
| Version: | 10.0 (Newton) | CC: | abishop, dhill, eharney, gsapienz, jobernar, srevivo, tshefi |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-07-10 22:50:27 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I don't have a access to VMAX storage.
When I tried to reproduce I migrated an attached volume(XtreamIO iscsi) to ceph I hit this error below[0] on Cinder volume log.
Unsure if it's related to this bug, or if new issues altogether.
An unattached volume migrated back and forth without issues.
[0]
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager [req-7371ed74-3fce-4c33-b452-de6fdfc5733b fc165990e8a14bf09c35b198067679d8 e530c8a5a45c4b3c8ec1df98d3a588c8 - default default] Failed to copy volume 802c7a19-fd05-42e7-b2da-d059e4c10777 to 634a11fa-5c45-4699-8d2b-98131a10faa9
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager Traceback (most recent call last):
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1895, in _migrate_volume_generic
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager new_volume.id)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/cinder/compute/nova.py", line 181, in update_server_volume
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager new_volume_id)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/novaclient/v2/volumes.py", line 68, in update_server_volume
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager body, "volumeAttachment")
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 374, in _update
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager resp, body = self.api.client.put(url, body=body)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 196, in put
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager return self.request(url, 'PUT', **kwargs)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 168, in request
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager **kwargs)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 344, in request
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 112, in request
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager return self.session.request(url, method, **kwargs)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 101, in inner
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager return wrapped(*args, **kwargs)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 486, in request
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager auth_headers = self.get_auth_headers(auth)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 757, in get_auth_headers
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager return auth.get_headers(self, **kwargs)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/plugin.py", line 90, in get_headers
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager token = self.get_token(session)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 90, in get_token
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager return self.get_access(session).auth_token
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 136, in get_access
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager self.auth_ref = self.get_auth_ref(session)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 179, in get_auth_ref
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager self._plugin = self._do_create_plugin(session)
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 174, in _do_create_plugin
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager raise exceptions.DiscoveryFailure('Could not determine a suitable URL '
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager DiscoveryFailure: Could not determine a suitable URL for the plugin
2018-07-10 10:06:18.122 203551 ERROR cinder.volume.manager
2018-07-10 10:06:18.127 203551 DEBUG oslo_messaging._drivers.amqpdriver [req-7371ed74-3fce-4c33-b452-de6fdfc5733b fc165990e8a14bf09c35b198067679d8 e530c8a5a45c4b3c8ec1df98d3a588c8 - default default] CAST unique_id: 662820c733fa4d66a309655fea51a10f exchange 'openstack' topic 'cinder-volume.hostgroup@tripleo_ceph' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:568
Yes you hit this issue because your environment is not properly configured. You can apply the same kind of configuration as mentionned in this BZ [1]. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1587875 Jon, can you comment on this? Is the issue that this sort of retype isn't supported when a volume is attached? This feature is not yet supported, but is an RFE targeted for OSP-15. It's also dependent on a new libvirt feature, per bug #1306562. *** This bug has been marked as a duplicate of bug 1293440 *** |
Description of problem: Unable to retype from a VMAX backend to ceph backend or vice-versa as we get this error message when nova tries to swap the disks: 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] result = proxy_call(self._autowrap, f, *args, **kwargs) 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] rv = execute(f, *args, **kwargs) 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] six.reraise(c, e, tb) 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] rv = meth(*args, **kwargs) 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3795, in defineXML 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self) 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] libvirtError: block copy still active: domain has active block job 2018-07-05 17:15:57.050 5597 ERROR nova.compute.manager [instance: 4730b09f-6700-4a54-a945-66fe4e7399ad] I've tried retyping from vmax to ceph but that flow might not be supported as we have to detach (?) the disk from the instance first, attach it to the controller and do the retype thing ... If I'm wrong here, no worries as the above error message is the reason one of the flow isn't supported. Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Have a ceph backend in cinder 2. Have a vmax backend in cinder 3. Create a test VM 4. Create a volume in ceph 5. Retype it to vmax (fails when trying to swap the disks) AND 1. Have a ceph backend in cinder 2. Have a vmax backend in cinder 3. Create a test VM 4. Create a volume in the vmax backend 5. Retype it to ceph (fails faster) Actual results: Fails Expected results: Succeeds Additional info: