Bug 1062908
| Summary: | Nova instance creation fails with libvirtError: Unable to add port tap9cbcd041-02 to OVS bridge br-int: Operation not permitted | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | shilpa <smanjara> | ||||
| Component: | openstack-nova | Assignee: | Ryan Hallisey <rhallise> | ||||
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Ami Jeain <ajeain> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 4.0 | CC: | ndipanov, sgordon, yeylon | ||||
| Target Milestone: | --- | Keywords: | ZStream | ||||
| Target Release: | 5.0 (RHEL 7) | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2014-07-02 15:30:30 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Created attachment 860883 [details]
Nova debug logs
Debug logs attached after recreating the problem with instance-id "b740efec-47e1-4b86-af7a-6475b347fce1"
I'm sorry that we never got to this. Given the age of this bug, I don't think we'll be able to reproduce this at this point. This also looks like something that we should just handle upstream as a bug in launchpad. If this is still something that you care about, please just file a bug upstream. |
Description of problem: Nova instance creation fails with libvirtError: Unable to add port tap9cbcd041-02 to OVS bridge br-int: Operation not permitted. It was a previously working setup. No external network has been setup. Only a test setup to test RHOS with glusterfs. Using 6.5 RHEL Version-Release number of selected component (if applicable): libvirt-0.10.2-29.el6_5.3.x86_64 libvirt-client-0.10.2-29.el6_5.3.x86_64 openstack-nova-2013.2.1-2.el6ost.noarch How reproducible: Ever since the first failure, always. Steps to reproduce: 1. Installed and configure glance, cinder and nova to use glusterfs volumes. 2. Create and delete several instances. After a few instances, creation of vm's started failing. # nova boot --flavor 2 --image 082e2e6a-1893-48f2-949c-3104a77b42b7 instance-1 +--------------------------------------+--------------------------------------+ | Property | Value | +--------------------------------------+--------------------------------------+ | OS-EXT-STS:task_state | scheduling | | image | cirros | | OS-EXT-STS:vm_state | building | | OS-EXT-SRV-ATTR:instance_name | instance-0000001d | | OS-SRV-USG:launched_at | None | | flavor | m1.small | | id | 1069b9fb-6ffa-4389-9508-727751d7aa87 | | security_groups | [{u'name': u'default'}] | | user_id | b6e5391e2ec542fb9a25c23a5648950f | | OS-DCF:diskConfig | MANUAL | | accessIPv4 | | | accessIPv6 | | | progress | 0 | | OS-EXT-STS:power_state | 0 | | OS-EXT-AZ:availability_zone | nova | | config_drive | | | status | BUILD | | updated | 2014-02-08T13:39:32Z | | hostId | | | OS-EXT-SRV-ATTR:host | None | | OS-SRV-USG:terminated_at | None | | key_name | None | | OS-EXT-SRV-ATTR:hypervisor_hostname | None | | name | instance-1 | | adminPass | C9gBF7p2vp42 | | tenant_id | ae458c0ee9864efca47906f9f3b7278d | | created | 2014-02-08T13:39:32Z | | os-extended-volumes:volumes_attached | [] | | metadata | {} | +--------------------------------------+--------------------------------------+ --------+---------------------+ # nova list +--------------------------------------+------------+--------+------------+-------------+---------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------------+--------+------------+-------------+---------------------+ | 1069b9fb-6ffa-4389-9508-727751d7aa87 | instance-1 | BUILD | spawning | NOSTATE | public=172.24.4.227 | +--------------------------------------+------------+--------+------------+-------------+---------------------+ # nova list +--------------------------------------+------------+--------+------------+-------------+---------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------------+--------+------------+-------------+---------------------+ | 1069b9fb-6ffa-4389-9508-727751d7aa87 | instance-1 | ERROR | None | NOSTATE | public=172.24.4.227 | +--------------------------------------+------------+--------+------------+-------------+---------------------+ Expected results: Instances should have booted successfully. Additional info: Logs from compute.log: 2014-02-08 19:09:41.800 24378 ERROR nova.compute.manager [req-2e945b46-b9f1-4013-b5bd-b6e751e3715f b6e5391e2ec542fb9a25c23a5648950f ae458c0ee9864efca47906f9f3b7278d] [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] Error: Unable to add port tap9cbcd041-02 to OVS bridge br-int: Operation not permitted 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] Traceback (most recent call last): 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1037, in _build_instance 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] set_access_ip=set_access_ip) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1420, in _spawn 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] LOG.exception(_('Instance failed to spawn'), instance=instance) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1417, in _spawn 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] block_device_info) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2080, in spawn 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] block_device_info, context=context) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3235, in _create_domain_and_network 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] domain = self._create_domain(xml, instance=instance, power_on=power_on) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3178, in _create_domain 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] domain.createWithFlags(launch_flags) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 187, in doit 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] result = proxy_call(self._autowrap, f, *args, **kwargs) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 147, in proxy_call 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] rv = execute(f,*args,**kwargs) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 76, in tworker 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] rv = meth(*args,**kwargs) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] File "/usr/lib64/python2.6/site-packages/libvirt.py", line 708, in createWithFlags 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2014-02-08 19:09:41.800 24378 TRACE nova.compute.manager [instance: 1069b9fb-6ffa-4389-9508-727751d7aa87] libvirtError: Unable to add port tap9cbcd041-02 to OVS bridge br-int: Operation not permitted Tried to restart libvirtd and nova services with no success. Have not found a workaround.