Bug 1148193

Summary:	Evacuate Fails 'Invalid state of instance files' using Ceph Ephemeral RBD
Product:	Red Hat OpenStack	Reporter:	Josh Durgin <jdurgin>
Component:	openstack-nova	Assignee:	Pádraig Brady <pbrady>
Status:	CLOSED WONTFIX	QA Contact:	Ami Jeain <ajeain>
Severity:	unspecified	Docs Contact:
Priority:	high
Version:	5.0 (RHEL 7)	CC:	antoffka, berrange, dasmith, kimi.zhang, ndipanov, pbrady, sbauza, sclewis, sferdjao, sgordon, vromanso, yeylon, yrabl
Target Milestone:	z4	Keywords:	ZStream
Target Release:	5.0 (RHEL 7)
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	openstack-nova-2014.2.1-2.el7ost	Doc Type:	Bug Fix
Doc Text:	Cause: The evacuate function didn't consider RBD storage as shared Consequence: The evacuate procedure would fail with RBD backed instanves Fix: RBD storage is now marked as shared Result: The evacuate function handles the shared storage attribute and thus now operates on RBD.	Story Points:	---
Clone Of:
Clones:	1174422 1174424 (view as bug list)		Environment:
Last Closed:	2015-04-10 15:31:06 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	743661, 1038706, 1111295, 1174422, 1174424

Description Josh Durgin 2014-09-30 22:13:31 UTC

Description of problem:

We can't seem to be able to evacuate instances from a failed compute node using shared storage. We are using Ceph Ephemeral RBD as the storage medium.

Version-Release number of selected component (if applicable):

Havana, Icehouse, Juno

How reproducible:
Always

Steps to Reproduce:

nova evacuate --on-shared-storage 6e2081ec-2723-43c7-a730-488bb863674c node-24
or
POST to http://ip-address:port/v2/tenant_id/servers/server_id/action with
{"evacuate":{"host":"node-24","onSharedStorage":1}}

Here is what shows up in the logs:

180>Jul 10 20:36:48 node-24 nova-nova.compute.manager AUDIT: Rebuilding instance
<179>Jul 10 20:36:48 node-24 nova-nova.compute.manager ERROR: Setting instance vm_state to ERROR
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5554, in _error_out_instance_on_exception
    yield
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2434, in rebuild_instance
    _("Invalid state of instance files on shared"
InvalidSharedStorage: Invalid state of instance files on shared storage
<179>Jul 10 20:36:49 node-24 nova-oslo.messaging.rpc.dispatcher ERROR: Exception during message handling: Invalid state of instance files on shared storage
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
    incoming.message))
  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)
  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
    result = getattr(endpoint, method)(ctxt, **new_args)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 393, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/server.py", line 139, in inner
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
    payload)
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
    return f(self, context, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 274, in decorated_function
    pass
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 260, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 327, in decorated_function
    function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 303, in decorated_function
    e, sys.exc_info())
  File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 290, in decorated_function
    return function(self, context, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2434, in rebuild_instance
    _("Invalid state of instance files on shared"
InvalidSharedStorage: Invalid state of instance files on shared storage

Comment 4 Lon Hohberger 2015-04-10 15:31:06 UTC

This was fixed in RHEL OSP 6.