Bug 1285185

Summary: Instance migration failed with ('Connection aborted.', BadStatusLine("''",))
Product: Red Hat OpenStack Reporter: Chen <cchen>
Component: python-cinderclientAssignee: Eric Harney <eharney>
Status: CLOSED WORKSFORME QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.0 (Juno)CC: eharney, jruzicka, sgotliv, yeylon
Target Milestone: ---   
Target Release: 8.0 (Liberty)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-17 13:48:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Chen 2015-11-25 08:13:18 UTC
Description of problem:

Instance migration failed with ('Connection aborted.', BadStatusLine("''",))

2015-11-20 20:49:59.404 20905 ERROR nova.compute.manager [req-2e0865e2-59f5-4d2b-b00c-7647488f9dcd None] [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de] Setting instance vm_state to ERROR
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de] Traceback (most recent call last):
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3948, in finish_resize
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     disk_info, image)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3900, in _finish_resize
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     context, instance, refresh_conn_info=True)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1989, in _get_instance_block_device_info
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     self.driver)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 414, in refresh_conn_infos
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     block_device_mapping)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 46, in wrapped
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     ret_val = method(obj, context, *args, **kwargs)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 281, in refresh_connection_info
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     connector)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 185, in wrapper
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     res = method(self, ctx, volume_id, *args, **kwargs)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 349, in initialize_connection
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     connector)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/cinderclient/v1/volumes.py", line 321, in initialize_connection
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     {'connector': connector})[1]['connection_info']
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/cinderclient/v1/volumes.py", line 250, in _action
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     return self.api.client.post(url, body=body)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 305, in post
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     return self._cs_request(url, 'POST', **kwargs)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]   File "/usr/lib/python2.7/site-packages/cinderclient/client.py", line 294, in _cs_request
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de]     raise exceptions.ConnectionError(msg)
2015-11-20 20:49:59.404 20905 TRACE nova.compute.manager [instance: e10f7e51-01d7-4bc6-96db-cb34c4b3a5de] CinderConnectionFailed: Connection to cinder host failed: Unable to establish connection: ('Connection aborted.', BadStatusLine("''",))


Version-Release number of selected component (if applicable):

python-cinderclient-1.1.1-1.el7ost.noarch

How reproducible:

10~20%

Steps to Reproduce:
1. 
2.
3.

Actual results:

The cinderclient failed to post an API request to cinder-api and as a result the migration failed.

Expected results:

The migration should be successful

Additional info:

Comment 2 Sergey Gotliv 2015-12-06 15:36:58 UTC
Chen,

You opened 3 different cases for the same partner. All 3 describe different issues related to connectivity between different OpenStack components. Nova can't communicate with Cinder, Cinder can't communicate with the VNX driver and so on.
We can either setup a quick call with the partner to check his environment or let someone on the field do that.