Bug 1090605

Summary: Instances failed to spawn due to: VirtualInterfaceCreateException: Virtual Interface creation failed
Product: [Community] RDO Reporter: wes hayutin <whayutin>
Component: openstack-packstackAssignee: Martin Magr <mmagr>
Status: CLOSED CURRENTRELEASE QA Contact: Toure Dunnon <tdunnon>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: aortega, apevec, beagles, dallan, derekh, dradez, ichavero, ihrachys, mnewby, ndipanov, oblaut, pbrady, rbryant, whayutin, yeylon
Target Milestone: ---Keywords: WorkAround
Target Release: Icehouse   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-packstack-2014.1.1-0.9.dev1055.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1092661 1094444 (view as bug list) Environment:
Last Closed: 2014-05-06 15:26:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1067211, 1092661, 1094444    
Attachments:
Description Flags
install and run logs none

Description wes hayutin 2014-04-23 17:53:51 UTC
Created attachment 889055 [details]
install and run logs

Description of problem:

python-neutron-2014.1-10.el6.noarch
openstack-neutron-2014.1-10.el6.noarch
openstack-neutron-openvswitch-2014.1-10.el6.noarch

python-nova-2014.1-2.el6.noarch
openstack-nova-api-2014.1-2.el6.noarch

openstack-packstack-2014.1.1-0.8.dev1045.el6.noarch
openstack-puppet-modules-2014.1-7.el6.noarch


/var/log/nova/compute.log
a6e1-0ebfdd5f76d1] Instance failed to spawn
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1] Traceback (most recent call last):
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1720, in _spawn
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1]     block_device_info)
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1]   File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2253, in spawn
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1]     block_device_info)
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1]   File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3663, in _create_domain_and_network
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1]     raise exception.VirtualInterfaceCreateException()
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1] VirtualInterfaceCreateException: Virtual Interface creation failed
2014-04-23 08:25:51.255 2735 TRACE nova.compute.manager [instance: 27f78140-6034-4e85-a6e1-0ebfdd5f76d1] 

Able to work around the issue by setting the following in nova.conf on the compute node.
 vif_plugging_is_fatal: false
 vif_plugging_timeout value: 0


Attaching install and run logs...

Comment 2 Alan Pevec 2014-04-29 16:24:46 UTC
From attached log archive:
 Apr 23 07:55:22 Installed: 1:python-novaclient-2.17.0-1.el6.noarch

Comment 3 Nikola Dipanov 2014-04-29 16:25:22 UTC
We need to have python-novaclient 2.17 (python-novaclient-2.17.0-1.el7.noarch.rpm) installed for this to work (seems it is from the logs), and also make sure that all the Neoutron configuration around sending events back to Nova is properly set up (see https://review.openstack.org/#/c/75253/29/neutron/common/config.py, the commit that introduces it for more details.)

In case the config is the problem - we need to clone to Neutron, as it needs to be fixed there

Comment 4 Ihar Hrachyshka 2014-04-29 16:33:45 UTC
Novaclient is fresh enough; the neutron configuration settings for nova are in Icehouse. So the only thing we probably miss is proper configuration. If that's the case, configuration is to be done by puppet modules. So there is no need to clone to neutron. Instead, clone to packstack.

Comment 5 Nikola Dipanov 2014-04-29 16:49:12 UTC
Yep - that is correct. Looking at the neutron.conf from the attached logs - it is missing all the needed conf from the the attached patch. Cloning to puppet modules.

Comment 6 Nikola Dipanov 2014-04-29 17:00:46 UTC
*** Bug 1092661 has been marked as a duplicate of this bug. ***

Comment 7 wes hayutin 2014-04-29 17:36:20 UTC
attached log:
/var/log/rpm.list

python-novaclient-2.17.0-1.el6.noarch

Comment 8 wes hayutin 2014-04-29 18:56:46 UTC
This may be part of the fix.. FYI

https://review.openstack.org/#/c/87009/

Comment 9 Brent Eagles 2014-04-29 19:05:47 UTC
That patch should do it. The defaults in nova enable the vif notifications, so nova should be fine. The missing neutron config (referenced in the patch in c8) should enable the neutron side notifications. The above-mentioned workaround worked because it turned of the nova side. As it would be preferable to use this feature, it would be better to fix the neutron config then disable it in nova.

While the linked patch should be sufficient for sussing out the required configuration, the comments in the "neutron nova interactions" area in neutron.conf describes the configuration options, etc.

Comment 10 Ivan Chavero 2014-04-30 15:23:11 UTC
The patch (https://review.openstack.org/#/c/87009/) has been merged

Comment 11 Martin Magr 2014-05-05 08:09:11 UTC
Moving this bug to correct product.