Bug 1480673

Summary: openstack-nova: failed to launch instance on OC: libvirtError: Cannot recv data: Connection reset by peer
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: openstack-novaAssignee: Eoghan Glynn <eglynn>
Status: CLOSED DUPLICATE QA Contact: Joe H. Rahme <jhakimra>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 12.0 (Pike)CC: berrange, dasmith, eglynn, kchamart, sbauza, sferdjao, sgordon, srevivo, vromanso
Target Milestone: rc   
Target Release: 12.0 (Pike)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-11 17:22:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2017-08-11 17:12:23 UTC
openstack-nova: failed to launch instance on OC: libvirtError: Cannot recv data: Connection reset by peer
Environment:
openstack-nova-scheduler-16.0.0-0.20170805120344.5971dde.el7ost.noarch
python-nova-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-console-16.0.0-0.20170805120344.5971dde.el7ost.noarch
puppet-nova-11.3.0-0.20170805105252.30a205c.el7ost.noarch
openstack-nova-conductor-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-novncproxy-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-compute-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-migration-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-api-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-common-16.0.0-0.20170805120344.5971dde.el7ost.noarch
python-novaclient-9.1.0-0.20170804194758.0a53d19.el7ost.noarch
openstack-nova-placement-api-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170805163045.el7ost.noarch
instack-undercloud-7.2.1-0.20170729010705.el7ost.noarch
openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch


Steps to reproduce:
Deploy OC with ceph and try to launch an instance

Result:
The instance gets status error:
(overcloud) [stack@undercloud-0 ~]$ nova list
+--------------------------------------+--------+--------+------------+-------------+----------+
| ID                                   | Name   | Status | Task State | Power State | Networks |
+--------------------------------------+--------+--------+------------+-------------+----------+
| 410a65ee-318b-471f-8d13-e904060d55ec | nisim1 | ERROR  | -          | NOSTATE     |          |
+--------------------------------------+--------+--------+------------+-------------+----------+
(overcloud) [stack@undercloud-0 ~]$ nova show nisim1|grep faul
| fault                                | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": "  File \"/usr/lib/python2.7/site-packages/nova
, line 553, in build_instances |
(overcloud) [stack@undercloud-0 ~]$ 


[root@overcloud-compute-0 ~]# grep -i error /var/log/containers/nova/nova-compute.log 
2017-08-11 16:58:08.822 1 ERROR nova.virt.libvirt.guest [req-469fe6ad-e339-4e11-a151-892fcc2bd53f 469f9aea9eb846a0bc14f5a6a417790a 06ed3e78a6b4443abd65a3a63aa492f8 - default default] Error launching a defined domain with XML: <domain type='kvm'>
: libvirtError: Cannot recv data: Connection reset by peer
2017-08-11 16:58:08.823 1 ERROR nova.virt.libvirt.driver [req-469fe6ad-e339-4e11-a151-892fcc2bd53f 469f9aea9eb846a0bc14f5a6a417790a 06ed3e78a6b4443abd65a3a63aa492f8 - default default] [instance: 410a65ee-318b-471f-8d13-e904060d55ec] Failed to start libvirt guest: libvirtError: Cannot recv data: Connection reset by peer
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [req-469fe6ad-e339-4e11-a151-892fcc2bd53f 469f9aea9eb846a0bc14f5a6a417790a 06ed3e78a6b4443abd65a3a63aa492f8 - default default] [instance: 410a65ee-318b-471f-8d13-e904060d55ec] Instance failed to spawn: libvirtError: Cannot recv data: Connection reset by peer
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec] Traceback (most recent call last):
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2125, in _build_resources
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     yield resources
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _build_and_run_instance
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     block_device_info=block_device_info)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2805, in spawn
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     destroy_disks_on_failure=True)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5274, in _create_domain_and_network
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     destroy_disks_on_failure)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     self.force_reraise()
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     six.reraise(self.type_, self.value, self.tb)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5244, in _create_domain_and_network
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     post_xml_callback=post_xml_callback)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5162, in _create_domain
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     guest.launch(pause=pause)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     self._encoded_xml, errors='ignore')
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     self.force_reraise()
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     six.reraise(self.type_, self.value, self.tb)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     return self._domain.createWithFlags(flags)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     rv = execute(f, *args, **kwargs)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     six.reraise(c, e, tb)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     rv = meth(*args, **kwargs)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1069, in createWithFlags
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec] libvirtError: Cannot recv data: Connection reset by peer
2017-08-11 16:58:09.898 1 ERROR nova.compute.manager [instance: 410a65ee-318b-471f-8d13-e904060d55ec]

Comment 1 Alexander Chuzhoy 2017-08-11 17:22:00 UTC

*** This bug has been marked as a duplicate of bug 1477720 ***