1374978 – Vms are not booting because virtlogd is not started by default after libvirt installation

Bug 1374978 - Vms are not booting because virtlogd is not started by default after libvirt installation

Summary: Vms are not booting because virtlogd is not started by default after libvirt ...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	distribution
Sub Component:
Version:	10.0 (Newton)
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	10.0 (Newton)
Assignee:	Lon Hohberger
QA Contact:	Prasanth Anbalagan
Docs Contact:
URL:
Whiteboard:
Depends On:	1325503 1372576
Blocks:
TreeView+	depends on / blocked

Reported:	2016-09-11 09:15 UTC by Alexander Stafeyev
Modified:	2019-09-09 15:50 UTC (History)
CC List:	18 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-12-14 16:00:40 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHEA-2016:2948	0	normal	SHIPPED_LIVE	Red Hat OpenStack Platform 10 enhancement update	2016-12-14 19:55:27 UTC

Description Alexander Stafeyev 2016-09-11 09:15:11 UTC

Description of problem:
VMs are failing to boot. 
libvirtError: Failed to connect socket to '/var/run/libvirt/virtlogd-sock': No such file or directory




Version-Release number of selected component (if applicable):
pack stack AIO+compute node deployment. 

openstack-nova-novncproxy-14.0.0-0.20160823074012.21312bf.el7ost.noarch
openstack-nova-scheduler-14.0.0-0.20160823074012.21312bf.el7ost.noarch
openstack-nova-cert-14.0.0-0.20160823074012.21312bf.el7ost.noarch
python-nova-14.0.0-0.20160823074012.21312bf.el7ost.noarch
openstack-nova-compute-14.0.0-0.20160823074012.21312bf.el7ost.noarch
openstack-nova-console-14.0.0-0.20160823074012.21312bf.el7ost.noarch
openstack-nova-conductor-14.0.0-0.20160823074012.21312bf.el7ost.noarch
openstack-nova-common-14.0.0-0.20160823074012.21312bf.el7ost.noarch
puppet-nova-9.1.0-0.20160823051658.5075d8b.el7ost.noarch
openstack-nova-api-14.0.0-0.20160823074012.21312bf.el7ost.noarch
python-novaclient-5.0.0-0.20160802172215.5eb7b65.el7ost.noarch

 rpm -qa | grep packstack 
openstack-packstack-puppet-8.0.0-0.20160818153549.c449b75.1.el7ost.noarch
openstack-packstack-8.0.0-0.20160818153549.c449b75.1.el7ost.noarch

!!!!!rhel 7.3!!!!!


rpm -qa | grep libvirt
libvirt-daemon-driver-nodedev-2.0.0-6.el7.x86_64
libvirt-daemon-driver-storage-2.0.0-6.el7.x86_64
libvirt-daemon-kvm-2.0.0-6.el7.x86_64
libvirt-daemon-driver-nwfilter-2.0.0-6.el7.x86_64
libvirt-client-2.0.0-6.el7.x86_64
libvirt-daemon-driver-interface-2.0.0-6.el7.x86_64
libvirt-daemon-driver-qemu-2.0.0-6.el7.x86_64
libvirt-daemon-config-nwfilter-2.0.0-6.el7.x86_64
libvirt-daemon-driver-secret-2.0.0-6.el7.x86_64
libvirt-daemon-2.0.0-6.el7.x86_64
libvirt-daemon-driver-network-2.0.0-6.el7.x86_64
libvirt-python-2.0.0-2.el7.x86_64

How reproducible:

100%
Steps to Reproduce:
1.delpoy rhos 10 with pack stack 
2.try to boot VMs
3.

Actual results:
Vm is in error state 

Expected results:
VM should be active

Additional info:
solved the issue: 
# systemctl start virtlogd
# systemctl enable  virtlogd



nova logs: 
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [req-7021700c-98aa-4dcf-ba5e-f68783361c15 72799ab7868043c4a4a83bafc489d51b c5b794afc5554b0183ffa6a588082d16 - - -] [instance: c48ef490-7558-426c-881e-62da0e80a897] Instance failed to spawn
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897] Traceback (most recent call last):
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2075, in _build_resources
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     yield resources
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1919, in _build_and_run_instance
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     block_device_info=block_device_info)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2583, in spawn
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     post_xml_callback=gen_confdrive)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4812, in _create_domain_and_network
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     post_xml_callback=post_xml_callback)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4741, in _create_domain
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     guest.launch(pause=pause)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     self._encoded_xml, errors='ignore')
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     self.force_reraise()
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     six.reraise(self.type_, self.value, self.tb)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     return self._domain.createWithFlags(flags)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     rv = execute(f, *args, **kwargs)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     six.reraise(c, e, tb)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     rv = meth(*args, **kwargs)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1065, in createWithFlags
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897] libvirtError: Failed to connect socket to '/var/run/libvirt/virtlogd-sock': No such file or directory
2016-09-11 09:29:32.067 7360 ERROR nova.compute.manager [instance: c48ef490-7558-426c-881e-62da0e80a897]
2016-09-11 09:29:32.068 7360 INFO nova.compute.manager [req-7021700c-98aa-4dcf-ba5e-f68783361c15 72799ab7868043c4a4a83bafc489d51b c5b794afc5554b0183ffa6a588082d16 - - -] [instance: c48ef490-7558-426c-881e-62da0e80a897] Terminating instance



same bug for fedora- https://bugzilla.redhat.com/show_bug.cgi?id=1290357

Comment 2 Daniel Berrangé 2016-09-12 08:34:22 UTC

Fixed by https://bugzilla.redhat.com/show_bug.cgi?id=1325503

Comment 3 Sahid Ferdjaoui 2016-09-12 08:45:12 UTC

We can't really handle that issue in Nova or the packaging, it's related to the installers. Not sure if packstack is officially supported for OSP10 or if it's just osp-director.

Comment 4 Mike Burns 2016-09-14 11:19:22 UTC

Looking at bug 1325503, this is fixed in RHEL.  There was a secondary issue in libvirt filed at bug 1372576 that should resolve the final piece of this issue.  

Marking this testonly and moving to ON_QA, though I certainly wouldn't argue if people want to close it as a duplicate of one of the other bugs.

Comment 10 Alexander Stafeyev 2016-09-27 05:48:56 UTC

do we need to wait until RC-1 to CDN, beta channel, that scheduled to Thu 2016-10-13 or the code already available

Comment 11 Prasanth Anbalagan 2016-10-05 09:15:56 UTC

No issues in RHOS10/RHEL7.3 setup as below. Please feel free to move bug to Verified.

[user@serverX ~]# 
[user@serverX ~]# rpm -qa | grep libvirt
libvirt-daemon-config-nwfilter-2.0.0-9.el7.x86_64
libvirt-daemon-driver-nwfilter-2.0.0-9.el7.x86_64
libvirt-daemon-driver-network-2.0.0-9.el7.x86_64
libvirt-daemon-driver-nodedev-2.0.0-9.el7.x86_64
libvirt-daemon-driver-secret-2.0.0-9.el7.x86_64
libvirt-client-2.0.0-9.el7.x86_64
libvirt-daemon-2.0.0-9.el7.x86_64
libvirt-daemon-driver-interface-2.0.0-9.el7.x86_64
libvirt-daemon-driver-storage-2.0.0-9.el7.x86_64
libvirt-python-2.0.0-2.el7.x86_64
libvirt-daemon-driver-qemu-2.0.0-9.el7.x86_64
libvirt-daemon-kvm-2.0.0-9.el7.x86_64

[user@serverX ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.3 Beta (Maipo)

[user@serverX ~]# 
[user@serverX ~]# yum list installed | grep openstack-nova
openstack-nova-api.noarch             1:14.0.0-0.20160929203854.59653c6.el7ost
openstack-nova-cert.noarch            1:14.0.0-0.20160929203854.59653c6.el7ost
openstack-nova-common.noarch          1:14.0.0-0.20160929203854.59653c6.el7ost
openstack-nova-compute.noarch         1:14.0.0-0.20160929203854.59653c6.el7ost
openstack-nova-conductor.noarch       1:14.0.0-0.20160929203854.59653c6.el7ost
openstack-nova-console.noarch         1:14.0.0-0.20160929203854.59653c6.el7ost
openstack-nova-novncproxy.noarch      1:14.0.0-0.20160929203854.59653c6.el7ost
openstack-nova-scheduler.noarch       1:14.0.0-0.20160929203854.59653c6.el7ost
[user@serverX ~]# 
[user@serverX ]# nova list --all
+--------------------------------------+-----------------------------+----------------------------------+--------+------------+-------------+---------------------+
| ID                                   | Name                        | Tenant ID                        | Status | Task State | Power State | Networks            |
+--------------------------------------+-----------------------------+----------------------------------+--------+------------+-------------+---------------------+
| a9b7d5ca-22e4-4b63-9d59-4a4a61aa852d | tempest-instance-1224273146 | 64fa95f6dcf74ee99231d49e6ae2fe15 | ACTIVE | -          | Running     | public=172.24.4.234 |
| 8ef95078-bd90-4f5b-b184-dcafd5eb3881 | tempest-instance-1608873158 | 7c23f669091c4eedbd0b753dd82717d2 | ACTIVE | -          | Running     | public=172.24.4.235 |
| ae57a4c2-93bc-4e0e-a2ef-16b18a973719 | tempest-instance-1961403439 | 0dc6ffaa785c46ecbe0fa59530463abb | ACTIVE | -          | Running     | public=172.24.4.238 |
| be74ec6f-a037-406f-b4c1-b12464546c7b | tempest-instance-1986880685 | f9d934a290be449989f8c8c5f936b712 | ACTIVE | -          | Running     | public=172.24.4.226 |
| 4d3ed454-e9df-465b-8be2-e69d22a9f7b2 | tempest-instance-452607875  | 37202087cfe847a4aa23a69006c9da09 | ACTIVE | -          | Running     | public=172.24.4.237 |
| 279e00e6-1c0b-4f0b-9568-22804b7d1216 | tempest-instance-871610042  | 36d0c89d541f4edb99ea260672f21076 | ACTIVE | -          | Running     | public=172.24.4.232 |
+--------------------------------------+-----------------------------+----------------------------------+--------+------------+-------------+---------------------+
[user@serverX ]#

Comment 14 errata-xmlrpc 2016-12-14 16:00:40 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html

Note You need to log in before you can comment on or make changes to this bug.