Description of problem: nova-compute service is not started after installation on Fedora 20 Version-Release number of selected component (if applicable): How reproducible: Yes Steps to Reproduce: 1. get latest packstack from github.com and use master branch 2. install http://repos.fedorapeople.org/repos/openstack/openstack-icehouse/fedora-20/rdo-release-icehouse-1.noarch.rpm 3. `packstack --allinone --os-heat-install=y` Actual results: [davidx@fedora ~(keystone_admin)]$ nova service-list +------------------+--------+----------+---------+-------+----------------------------+-----------------+ | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +------------------+--------+----------+---------+-------+----------------------------+-----------------+ | nova-consoleauth | fedora | internal | enabled | up | 2014-01-21T14:24:18.000000 | None | | nova-scheduler | fedora | internal | enabled | up | 2014-01-21T14:24:19.000000 | None | | nova-conductor | fedora | internal | enabled | up | 2014-01-21T14:24:18.000000 | None | | nova-compute | fedora | nova | enabled | down | None | None | | nova-cert | fedora | internal | enabled | up | 2014-01-21T14:24:18.000000 | None | +------------------+--------+----------+---------+-------+----------------------------+-----------------+ Expected results: nova-compute is started Additional info: cat /var/log/nova/compute.log: 2014-01-21 22:22:40.755 2042 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `tsc-deadline' specified more than once 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup Traceback (most recent call last): 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup x.wait() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup return self.thread.wait() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup return self._exit_event.wait() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup return self.greenlet.switch() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup service.start() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context()) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes()) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__ 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup self.update_status() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities() 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self) 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `tsc-deadline' specified more than once 2014-01-21 22:22:40.755 2042 TRACE nova.openstack.common.threadgroup [davidx@fedora ~(keystone_admin)]$
We have just recently created the stable/havana branch. Can you try with havana instead of icehouse ?
I see a similar problem happening in upstream Icehouse with Fedora 20 as well. This works just fine with Fedora 19, so I suspect something package-wise must be different: 2014-01-22 17:57:29.264 25388 DEBUG nova.virt.libvirt.config [-] Generated XML ('<cpu>\n <arch>x86_64</arch>\n <model>Westmere</model>\n <vendor>Intel</vendor>\n <topology sockets="2" cores="1" threads="1"/>\n <feature name="rdtscp"/>\n <feature name="hypervisor"/>\n <feature name="avx"/>\n <feature name="osxsave"/>\n <feature name="xsave"/>\n <feature name="x2apic"/>\n <feature name="pcid"/>\n <feature name="vmx"/>\n <feature name="pclmuldq"/>\n <feature name="ss"/>\n <feature name="ds"/>\n <feature name="vme"/>\n</cpu>\n',) to_xml /opt/stack/nova/nova/virt/libvirt/config.py:73 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 346, in fire_timers timer() File "/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__ cb(*args, **kw) File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 163, in _do_send waiter.switch(result) File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main result = function(*args, **kwargs) File "/opt/stack/nova/nova/openstack/common/service.py", line 480, in run_service service.start() File "/opt/stack/nova/nova/service.py", line 182, in start self.manager.pre_start_hook() File "/opt/stack/nova/nova/compute/manager.py", line 815, in pre_start_hook self.update_available_resource(nova.context.get_admin_context()) File "/opt/stack/nova/nova/compute/manager.py", line 5012, in update_available_resource nodenames = set(self.driver.get_available_nodes()) File "/opt/stack/nova/nova/virt/driver.py", line 1058, in get_available_nodes stats = self.get_host_stats(refresh=refresh) File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4573, in get_host_stats return self.host_state.get_host_stats(refresh=refresh) File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 424, in host_state self._host_state = HostState(self) File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4964, in __init__ self.update_status() File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5003, in update_status self.driver.get_instance_capabilities() File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3697, in get_instance_capabilities caps = self.get_host_capabilities() File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2765, in get_host_capabilities libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES) File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit result = proxy_call(self._autowrap, f, *args, **kwargs) File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call rv = execute(f,*args,**kwargs) File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker rv = meth(*args,**kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self) libvirtError: internal error: CPU feature `tsc-deadline' specified more than once 2014-01-22 17:57:29.280 25388 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `tsc-deadline' specified more than once 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup Traceback (most recent call last): 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 117, in wait 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup x.wait() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/threadgroup.py", line 49, in wait 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup return self.thread.wait() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup return self._exit_event.wait() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup return hubs.get_hub().switch() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup return self.greenlet.switch() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/service.py", line 480, in run_service 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup service.start() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/service.py", line 182, in start 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup self.manager.pre_start_hook() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 815, in pre_start_hook 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup self.update_available_resource(nova.context.get_admin_context()) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 5012, in update_available_resource 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup nodenames = set(self.driver.get_available_nodes()) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/driver.py", line 1058, in get_available_nodes 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup stats = self.get_host_stats(refresh=refresh) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4573, in get_host_stats 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup return self.host_state.get_host_stats(refresh=refresh) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 424, in host_state 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup self._host_state = HostState(self) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4964, in __init__ 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup self.update_status() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5003, in update_status 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup self.driver.get_instance_capabilities() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3697, in get_instance_capabilities 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup caps = self.get_host_capabilities() 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2765, in get_host_capabilities 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup rv = execute(f,*args,**kwargs) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup rv = meth(*args,**kwargs) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self) 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `tsc-deadline' specified more than once 2014-01-22 17:57:29.280 25388 TRACE nova.openstack.common.threadgroup [kmestery@fedora-mac tempest]$ [kmestery@fedora-mac devstack]$
I did try Havana, but I got another issue about nova-network and logged it on launchpad: https://bugs.launchpad.net/packstack/+bug/1271096 Should I log that network issue on bugzilla?
Dan, do you have any idea? This is blocking our OpenDaylight work on F20.
It seems to be a libvirtd bug as mentioned in this bug over here too https://bugs.launchpad.net/nova/+bug/1267191 and here bug 1049391 even if the cpu flag is different in their case.
This is a duplicate of the below bug, where you could see the root-cause identified: https://bugzilla.redhat.com/show_bug.cgi?id=1049391#c11 And, related upstream bug: https://bugs.launchpad.net/nova/+bug/1267191
*** This bug has been marked as a duplicate of bug 1049391 ***