Hide Forgot
Created attachment 1121362 [details] logs Description of problem: A cinder volume does not detach from instance . Version-Release number of selected component (if applicable): python-novaclient-2.17.0-2.el7ost.noarch openstack-nova-api-2014.1.5-1.el7ost.noarch openstack-nova-common-2014.1.5-1.el7ost.noarch openstack-nova-compute-2014.1.5-1.el7ost.noarch python-cinderclient-1.0.9-1.el7ost.noarch How reproducible: No Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: We notice the following error in nova-compute.log : 2016-02-05 10:39:30.445 2157 AUDIT nova.compute.manager [req-af08d252-60a7-4d33-bc9e-f4d622a1175c 362ebe22e95942a9a15e434ef8225abf 8c67b3e3a71e4a878a2f0e78384c4149] [instance: 87ae0381-3dd8-4b17-b8f6-b99773e7d79 0] Detach volume 0ef814b8-09f7-470b-a162-a89773f01a51 from mountpoint /dev/vdb 2016-02-05 10:39:30.548 2157 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: expected string or buffer 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher payload) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__ 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher return f(self, context, *args, **kw) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 293, in decorated_function 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher LOG.warning(msg, e, instance_uuid=instance_uuid) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__ 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 270, in decorated_function 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 322, in decorated_function 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher e, sys.exc_info()) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__ 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 309, in decorated_function 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4364, in detach_volume 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher self._detach_volume(context, instance, bdm) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4311, in _detach_volume 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher connection_info = jsonutils.loads(bdm.connection_info) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/openstack/common/jsonutils.py", line 164, in loads 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher return json.loads(s) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher return _default_decoder.decode(s) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher TypeError: expected string or buffer 2016-02-05 10:39:30.548 2157 TRACE oslo.messaging.rpc.dispatcher